OF packages: check OF repositories

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

OF packages: check OF repositories

Olaf Till-2
Hi,

I'd suggest that OF package maintainers check that their packages
repositories at Sourceforge are up-to-date. It seems that in some of
them the last changesets are missing (again). (Last time this happend
Sourceforge said they installed an older repository backup after a
repository corruption.)

Olaf

--
public key id EAFE0591, e.g. on x-hkp://pool.sks-keyservers.net

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: OF packages: check OF repositories

Doug Stewart-4


On Sun, Apr 14, 2019 at 1:37 PM Olaf Till <[hidden email]> wrote:
Hi,

I'd suggest that OF package maintainers check that their packages
repositories at Sourceforge are up-to-date. It seems that in some of
them the last changesets are missing (again). (Last time this happend
Sourceforge said they installed an older repository backup after a
repository corruption.)

Olaf

--
public key id EAFE0591, e.g. on x-hkp://pool.sks-keyservers.net

the control pkg is definitely Not up to date!!!

I will push when we are sure that the problems are  fixed.

--
DASCertificate for 206392

Reply | Threaded
Open this post in threaded view
|

Re: OF packages: check OF repositories

Nir Krakauer-3
Can we make a plan to regularly back up the whole OF tree on another site to protect against this kind of eventuality?
Reply | Threaded
Open this post in threaded view
|

Re: OF packages: check OF repositories

Juan Pablo Carbajal-2
Hi,

geometry was sent back a year into the past.
I just pushed.

Thanks for the heads up

On Sun, Apr 14, 2019 at 9:34 PM Nir Krakauer <[hidden email]> wrote:
>
> Can we make a plan to regularly back up the whole OF tree on another site to protect against this kind of eventuality?

Reply | Threaded
Open this post in threaded view
|

Re: OF packages: check OF repositories

Doug Stewart-4
I pushed control

On Sun, Apr 14, 2019 at 4:10 PM Juan Pablo Carbajal <[hidden email]> wrote:
Hi,

geometry was sent back a year into the past.
I just pushed.

Thanks for the heads up

On Sun, Apr 14, 2019 at 9:34 PM Nir Krakauer <[hidden email]> wrote:
>
> Can we make a plan to regularly back up the whole OF tree on another site to protect against this kind of eventuality?


--
DASCertificate for 206392

Reply | Threaded
Open this post in threaded view
|

Re: OF packages: check OF repositories

siko1056
On Mon, Apr 15, 2019 at 5:12 AM Doug Stewart <[hidden email]> wrote:
I pushed control

On Sun, Apr 14, 2019 at 4:10 PM Juan Pablo Carbajal <[hidden email]> wrote:
Hi,

geometry was sent back a year into the past.
I just pushed.

Thanks for the heads up

On Sun, Apr 14, 2019 at 9:34 PM Nir Krakauer <[hidden email]> wrote:
>
> Can we make a plan to regularly back up the whole OF tree on another site to protect against this kind of eventuality?


--
DASCertificate for 206392


SourceForge is very unreliable and annoying these days.  After the release of the struct package (special thanks to Olaf), I was now able to install the optim package in Octave 5.1.0, with a small fix from bug #54487 [1].  But I was not able to watch out online or by cloning the entire repo for other important changes since the 1.5.3 release, because the optim repo website is down (error reported at SF [2]) and I have no repo URL available at [3].  Additionally I was afraid, that even if I had this URL for cloning, the commits are not up-to-date  -.-

Maybe backing up/moving everything in Savannah?  Even if it's interface looks like 10 years unmaintained, it has no commercials, and is more reliable.  My personal favorite is BitBucket (it handles Mercurial repositories as well).

Kai

Reply | Threaded
Open this post in threaded view
|

RE: OF packages: check OF repositories

JohnD
In reply to this post by Olaf Till-2
>
> Message: 1
> Date: Sun, 14 Apr 2019 19:37:22 +0200
> From: Olaf Till <[hidden email]>
> To: Octave Maintainers <[hidden email]>
> Subject: OF packages: check OF repositories
> Message-ID: <20190414173722.pgrdqujrvyrw4wgh@till>
> Content-Type: text/plain; charset="us-ascii"
>
> Hi,
>
> I'd suggest that OF package maintainers check that their packages
> repositories at Sourceforge are up-to-date. It seems that in some of
> them the last changesets are missing (again). (Last time this happend
> Sourceforge said they installed an older repository backup after a
> repository corruption.)
>
> Olaf
>

I went to check windows, arduino and dicom, however they currently appear
broken when trying to pull or push to them.
Checking them on the web browser reports them as empty repos.

I have a ticket in https://sourceforge.net/p/forge/site-support/19285/  and
just got a response back:

(from SF)
Hi,
Yes, can you please re-push latest commits to the following octave repos:
arduino
dicom
image
instrument-control
mapping
optim
simp
statistics
stk
windows
zeromq
We will be moving these repositories to a different filesystem server very
soon to avoid these errors from happening again.
Sincerely,
SourceForge Support






Reply | Threaded
Open this post in threaded view
|

RE: OF packages: check OF repositories

JohnD


> -----Original Message-----
> From: JohnD [mailto:[hidden email]]
> Sent: Monday, April 15, 2019 11:02 AM
> To: [hidden email]
> Cc: 'Olaf Till'
> Subject: RE: OF packages: check OF repositories
>
> >
> > Message: 1
> > Date: Sun, 14 Apr 2019 19:37:22 +0200
> > From: Olaf Till <[hidden email]>
> > To: Octave Maintainers <[hidden email]>
> > Subject: OF packages: check OF repositories
> > Message-ID: <20190414173722.pgrdqujrvyrw4wgh@till>
> > Content-Type: text/plain; charset="us-ascii"
> >
> > Hi,
> >
> > I'd suggest that OF package maintainers check that their packages
> > repositories at Sourceforge are up-to-date. It seems that in some of
> > them the last changesets are missing (again). (Last time this happend
> > Sourceforge said they installed an older repository backup after a
> > repository corruption.)
> >
> > Olaf
> >
>
> I went to check windows, arduino and dicom, however they currently appear
> broken when trying to pull or push to them.
> Checking them on the web browser reports them as empty repos.
>
> I have a ticket in https://sourceforge.net/p/forge/site-support/19285/
and

> just got a response back:
>
> (from SF)
> Hi,
> Yes, can you please re-push latest commits to the following octave repos:
> arduino
> dicom
> image
> instrument-control
> mapping
> optim
> simp
> statistics
> stk
> windows
> zeromq
> We will be moving these repositories to a different filesystem server very
> soon to avoid these errors from happening again.
> Sincerely,
> SourceForge Support
>
>
>
>


Of the mentioned ones, I pushed back up arduino, dicom, instrument-control,
statistics, windows, zeromq

However, I did not have the tag push for the statistics release, so Olaf can
you repush it ?

I also checked the other repos I am currently maintaining: cgi, divand,
general, ncarray, netcdf and optiminterp - they appear to be up to date


Reply | Threaded
Open this post in threaded view
|

Re: OF packages: check OF repositories

Carnë Draug
In reply to this post by Olaf Till-2
On Mon, 15 Apr 2019 at 16:02, JohnD wrote:

> On Sun, 14 Apr 2019 at 19:37, Olaf Till wrote:
>> Hi,
>>
>> I'd suggest that OF package maintainers check that their packages
>> repositories at Sourceforge are up-to-date. It seems that in some of
>> them the last changesets are missing (again). (Last time this happend
>> Sourceforge said they installed an older repository backup after a
>> repository corruption.)
>>
>
> I went to check windows, arduino and dicom, however they currently appear
> broken when trying to pull or push to them.
> Checking them on the web browser reports them as empty repos.
>
> I have a ticket in https://sourceforge.net/p/forge/site-support/19285/  and
> just got a response back:
>
> (from SF)
> Hi,
> Yes, can you please re-push latest commits to the following octave repos:
> arduino
> dicom
> image
> instrument-control
> mapping
> optim
> simp
> statistics
> stk
> windows
> zeromq
> We will be moving these repositories to a different filesystem server very
> soon to avoid these errors from happening again.

I have pushed 'image'.  This I know my clone was up to date.

I have also pushed what I had of 'optim', 'mapping', 'simp', and 'stk'
but these may not have been the latest commits.  If someone else has a
backup of them, please take a look again.

David

Reply | Threaded
Open this post in threaded view
|

Re: OF packages: check OF repositories

PhilipNienhuis
Carnë Draug wrote

> On Mon, 15 Apr 2019 at 16:02, JohnD wrote:
>> On Sun, 14 Apr 2019 at 19:37, Olaf Till wrote:
>>> Hi,
>>>
>>> I'd suggest that OF package maintainers check that their packages
>>> repositories at Sourceforge are up-to-date. It seems that in some of
>>> them the last changesets are missing (again). (Last time this happend
>>> Sourceforge said they installed an older repository backup after a
>>> repository corruption.)
>>>
>>
>> I went to check windows, arduino and dicom, however they currently appear
>> broken when trying to pull or push to them.
>> Checking them on the web browser reports them as empty repos.
>>
>> I have a ticket in https://sourceforge.net/p/forge/site-support/19285/ 
>> and
>> just got a response back:
>>
>> (from SF)
>> Hi,
>> Yes, can you please re-push latest commits to the following octave repos:
>> arduino
>> dicom
>> image
>> instrument-control
>> mapping
>> optim
>> simp
>> statistics
>> stk
>> windows
>> zeromq
>> We will be moving these repositories to a different filesystem server
>> very
>> soon to avoid these errors from happening again.
>
> I have pushed 'image'.  This I know my clone was up to date.
>
> I have also pushed what I had of 'optim', 'mapping', 'simp', and 'stk'
> but these may not have been the latest commits.  If someone else has a
> backup of them, please take a look again.
>
> David

David?

Anyway I pushed io and mapping seems to be OK now.

Philip




--
Sent from: http://octave.1599824.n4.nabble.com/Octave-Maintainers-f1638794.html

Reply | Threaded
Open this post in threaded view
|

Re: OF packages: check OF repositories

Olaf Till-2
In reply to this post by JohnD
On Mon, Apr 15, 2019 at 11:22:37AM -0400, JohnD wrote:

>
>
> > -----Original Message-----
> > From: JohnD [mailto:[hidden email]]
> > Sent: Monday, April 15, 2019 11:02 AM
> > To: [hidden email]
> > Cc: 'Olaf Till'
> > Subject: RE: OF packages: check OF repositories
> >
> > >
> > > Message: 1
> > > Date: Sun, 14 Apr 2019 19:37:22 +0200
> > > From: Olaf Till <[hidden email]>
> > > To: Octave Maintainers <[hidden email]>
> > > Subject: OF packages: check OF repositories
> > > Message-ID: <20190414173722.pgrdqujrvyrw4wgh@till>
> > > Content-Type: text/plain; charset="us-ascii"
> > >
> > > Hi,
> > >
> > > I'd suggest that OF package maintainers check that their packages
> > > repositories at Sourceforge are up-to-date. It seems that in some of
> > > them the last changesets are missing (again). (Last time this happend
> > > Sourceforge said they installed an older repository backup after a
> > > repository corruption.)
> > >
> > > Olaf
> > >
> >
> > I went to check windows, arduino and dicom, however they currently appear
> > broken when trying to pull or push to them.
> > Checking them on the web browser reports them as empty repos.
> >
> > I have a ticket in https://sourceforge.net/p/forge/site-support/19285/
> and
> > just got a response back:
> >
> > (from SF)
> > Hi,
> > Yes, can you please re-push latest commits to the following octave repos:
> > arduino
> > dicom
> > image
> > instrument-control
> > mapping
> > optim
> > simp
> > statistics
> > stk
> > windows
> > zeromq
> > We will be moving these repositories to a different filesystem server very
> > soon to avoid these errors from happening again.
> > Sincerely,
> > SourceForge Support
> >
> >
> >
> >
>
>
> Of the mentioned ones, I pushed back up arduino, dicom, instrument-control,
> statistics, windows, zeromq
>
> However, I did not have the tag push for the statistics release, so Olaf can
> you repush it ?
Yes, done. There were two more, between the released changeset and the
tag push.

database, parallel, struct are refreshed, also optim, which came back
online with the negligible number of 86 changesets lacking.

This leaves mapping and simp, according to Sourceforge, but we heard
from Doug and JuanP that there were (are?) more.

Olaf

--
public key id EAFE0591, e.g. on x-hkp://pool.sks-keyservers.net

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: OF packages: check OF repositories

Olaf Till-2
In reply to this post by Nir Krakauer-3
On Sun, Apr 14, 2019 at 03:29:44PM -0400, Nir Krakauer wrote:
> Can we make a plan to regularly back up the whole OF tree on another site
> to protect against this kind of eventuality?

As for being able to restore, all repositories fit well onto a usual
local disk. And there is (or was) already a script to automatize the
backup of all repositories (a Makefile made by Carnë) in an extra
repository named 'octave-forge'. It only needs someone who regularly
thinks of it or scripts it as a cron job.

Olaf

--
public key id EAFE0591, e.g. on x-hkp://pool.sks-keyservers.net

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: OF packages: check OF repositories

Doug Stewart-4


On Mon, Apr 15, 2019 at 4:54 PM Olaf Till <[hidden email]> wrote:
On Sun, Apr 14, 2019 at 03:29:44PM -0400, Nir Krakauer wrote:
> Can we make a plan to regularly back up the whole OF tree on another site
> to protect against this kind of eventuality?

As for being able to restore, all repositories fit well onto a usual
local disk. And there is (or was) already a script to automatize the
backup of all repositories (a Makefile made by Carnë) in an extra
repository named 'octave-forge'. It only needs someone who regularly
thinks of it or scripts it as a cron job.

Olaf

--
public key id EAFE0591, e.g. on x-hkp://pool.sks-keyservers.net

OK I just ran Carne's make file, and have a copy of what is there now.
I will try and keep it up to date.
Someone else should also do it. then we might be safe. :-)

--
DASCertificate for 206392

Reply | Threaded
Open this post in threaded view
|

Re: OF packages: check OF repositories

siko1056
On Tue, Apr 16, 2019 at 7:25 AM Doug Stewart <[hidden email]> wrote:
On Mon, Apr 15, 2019 at 4:54 PM Olaf Till <[hidden email]> wrote:
On Sun, Apr 14, 2019 at 03:29:44PM -0400, Nir Krakauer wrote:
> Can we make a plan to regularly back up the whole OF tree on another site
> to protect against this kind of eventuality?

As for being able to restore, all repositories fit well onto a usual
local disk. And there is (or was) already a script to automatize the
backup of all repositories (a Makefile made by Carnë) in an extra
repository named 'octave-forge'. It only needs someone who regularly
thinks of it or scripts it as a cron job.

Olaf
OK I just ran Carne's make file, and have a copy of what is there now.
I will try and keep it up to date.
Someone else should also do it. then we might be safe. :-)

--
DAS

Just for the files, Carnës Makefile is available from https://sourceforge.net/p/octave/octave-forge/ci/default/tree/ or via

   hg clone http://hg.code.sf.net/p/octave/octave-forge octave-octave-forge

After cloning and running `make update` and about 220 MB with backups of 121 hg and git repositories from SF are downloaded in that very same directory.

As this backup is not that big, maybe we can run it daily via cron on JWEs digitalocean server, where other backups are located as well:


Ideally, there was public web access to those backups like with the other https://hg.octave.org repositories.
This backup can be also done on other server I have access to (JWEs dreamhost server):


But I am afraid that the backup repositories are not public visible there.

Best,
Kai
Reply | Threaded
Open this post in threaded view
|

Re: OF packages: check OF repositories

apjanke-floss


On 4/15/19 11:14 PM, Kai Torben Ohlhus wrote:

> On Tue, Apr 16, 2019 at 7:25 AM Doug Stewart <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     On Mon, Apr 15, 2019 at 4:54 PM Olaf Till <[hidden email]
>     <mailto:[hidden email]>> wrote:
>
>         On Sun, Apr 14, 2019 at 03:29:44PM -0400, Nir Krakauer wrote:
>         > Can we make a plan to regularly back up the whole OF tree on
>         another site
>         > to protect against this kind of eventuality?
>
>         As for being able to restore, all repositories fit well onto a usual
>         local disk. And there is (or was) already a script to automatize the
>         backup of all repositories (a Makefile made by Carnë) in an extra
>         repository named 'octave-forge'. It only needs someone who regularly
>         thinks of it or scripts it as a cron job.
>
>         Olaf
>
>     OK I just ran Carne's make file, and have a copy of what is there now.
>     I will try and keep it up to date.
>     Someone else should also do it. then we might be safe. :-)
>
>     --
>     DAS
>
>
> Just for the files, Carnës Makefile is available
> from https://sourceforge.net/p/octave/octave-forge/ci/default/tree/ or via
>
>    hg clone http://hg.code.sf.net/p/octave/octave-forge octave-octave-forge
>
> After cloning and running `make update` and about 220 MB with backups of
> 121 hg and git repositories from SF are downloaded in that very same
> directory.
>
> As this backup is not that big, maybe we can run it daily via cron on
> JWEs digitalocean server, where other backups are located as well:
>
>  
>  https://wiki.octave.org/Project_Infrastructure#Hosted_on_jwe.27s_digitalocean.com_account
>
> Ideally, there was public web access to those backups like with the
> other https://hg.octave.org repositories.
> This backup can be also done on other server I have access to (JWEs
> dreamhost server):
>
>  
>  https://wiki.octave.org/Project_Infrastructure#Hosted_on_jwe.27s_dreamhost.com_account
>
> But I am afraid that the backup repositories are not public visible there.
>
> Best,
> Kai

I've written a script to go with Carnë's Makefile that will mirror all
the OF repos to Bitbucket:

https://bitbucket.org/apjanke/octave-octave-forge/branch/bitbucket-sync

It's best used with a separate Bitbucket account so you don't clutter up
your main Bitbucket account with these mirrors, or get them confused
with your working forks of them. You'll need to set up an SSH key and
password for the Bitbucket account. Then you can `make update; make
bbpush` to update the mirror.

Here's what it looks like when they're all mirrored:
https://bitbucket.org/apjanke-octave/

Bitbucket is free (as in beer), the repos are publicly available, and
they have their own backups. This might be a good option.

The programmatic creation of Git repos via Bitbucket's REST API doesn't
seem to be working, so those need to be created manually, but there's
only a dozen of them, so that's not that much work. I've reported this
issue to Bitbucket.

Cheers,
Andrew