Re: Octave date and time services

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Octave date and time services

Mike Miller-4
On Fri, May 11, 2018 at 11:57:14 +0200, Ian McCallion wrote:
> I recently posed a question about timezones on the list that received
> no reply. No big deal, I solved my own problem and recognise this is
> probably not an area that concerns most of the Octave community.

I haven't seen your previous message. I am happy to try to address some
of the issues you bring up here. Maybe I can reply briefly to each one
and we can move this to octave-maintainers or to individual bug reports?

> 1. Documentation problem. Documentation for gmtime,
> https://octave.org/doc/v4.4.0/Timing-Utilities.html#XREFgmtime, says:
>
>     "Given a value returned from time, or any non-negative integer.
> return a time structure
>     corresponding to CUT (Coordinated Universal Time)."
>
> What gmtime really does is
>
>    "Given the number (>=0) of seconds since 00:00 on 1970/01/01(*) of
> an event, returns
>    a time structure describing that time of that event in UTC.
>
>    (*) This measure of recent and future time referred to as Seconds
> Since Epoch and is
>         returned by time(), lstat(filename), and other functions.
>
> This lack of precision applies to localtime and other functions also.
These basically read the same to me. When we provide a wrapper for a C
standard library function, we typically expect that the user already
knows how to use the C function. I would rather see Octave include
something like "for more details, refer to the C language reference
definition of gmtime".

> 2. Bug. In the tm_struct returned by the localtime function, the
> gmtoff field does not appear to be set correctly.

Works for me. Can you give an example?

> 3. Bug (in code or documentation). The mktime function which takes a
> tm_struct as parameter  and returns Seconds Since Epoch silently
> ignores the gmtoff and zone elements of the tm_struct. See also point
> 4.

Correct, those fields are informational extension fields for the user,
and are not used in the inverse operation. This is consistent with the C
function.

> 4. Bug or Omission. This function, which I wrote after much trial and
> error to solve my original problem:
>
>   function [] = settimezone(tz)
>       % Provide access to timezone conversions using mktime and localtime
>       if isempty(tz)  % Use the system's timezone
>            unsetenv("TZ");
>       else
>            setenv("TZ", tz);
>       end
>       mktime(localtime(1)); % Force Octave to access the TZ environment variable
>   end
>
> sets the timezone for the running octave environment according to the
> supplied timezone string, eg "EST+5EDT,M3.2.0/2,M11.1.0/2". The
> function is necessary because merely setting TZ is not sufficient -
> localtime and maybe other functions seem to reference a cache that is
> only loaded by mktime.
Does the following not work for you?

    x = time ();
    t1 = localtime (x)
    setenv TZ UTC0
    t2 = localtime (x)
    setenv TZ CST6CDT
    t3 = localtime (x)

If not, that may be a defect in your operating system, it works for me.

> The definition of timezone strings is available here:
>
>     https://www.gnu.org/software/libc/manual/html_node/TZ-Variable.html#TZ-Variable
>
> This should be be included into Octave documentation. There are also
> theoretically ways of referencing the timezone such as
> TZ=":America/New_York" which I have not been able to make work on
> windows (and I have no access to unix systems)

Any TZ setting that starts with a colon ':' is system-dependent, so it's
no surprise that it doesn't work on Windows, or may use different
definitions.

I don't think it's within the scope of the Octave documentation to
describe the TZ environment variable. I think it would be appropriate to
refer to the section in the GNU libc manual.

> 5. Bug in the Octave GUI editor. On Windows setting TZ does not affect
> file system APIs. So when the editor saves a function file the
> system's UTC clock is naturally used to timestamp the file. However
> when the editor loads a function file it (I am assuming) erroneously
> translates the file's timestamp to local time using the TZ variable
> but takes the local time from system APIs. The result is that the a
> file just saved can be reported as having a timestamp in the future.

Yes that may be a bug.

> 6. Bug, or documentation error. "Leap Seconds" are mentioned as
> ignored in text under the datenum function, but leap seconds apply to
> many APIs and what "ignored" means is ambiguous. If it is decided that
> a particular day will have 86401 atomic clock-defined seconds does
> "ignored" mean the seconds counter ranges 0:86400 instead of 0:86399?
> Or does it mean the clock is ever after out of step with the world's
> rotation and almost all other clocks in the world? Or does it mean the
> system on which Octave is running has slowed its timing services for a
> few hours or days to remain in sync with the accepted world standard
> time.
It simply means that in the context of the datenum function, every day
is exactly 86400 seconds. The following two expressions return the same
datenum value:

    datenum (2016, 12, 31, 15, 59, 60)
    datenum (2016, 12, 31, 16, 00, 00)

in my local time zone UTC-08:00.

--
mike

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Octave date and time services

Ian McCallion
Hi Mike,

Thanks for your reply. I have embedded answers to your questions below
and expanded a couple of my arguments. I don't want to get into a long
discussion, as I'm sure you don't either so I think I can best serve
Octave by writing up the bug reports you would like written up and the
next step should be for you to say which they are.

Cheers... Ian
----------------
On 11 May 2018 at 19:17, Mike Miller <[hidden email]> wrote:
> On Fri, May 11, 2018 at 11:57:14 +0200, Ian McCallion wrote:
>> <snip>
> Maybe I can reply briefly to each one
> and we can move this to octave-maintainers or to individual bug reports?

Yes. Happy with that and I have subscribed to octave-maintainers.

>> 2. Bug. In the tm_struct returned by the localtime function, the
>> gmtoff field does not appear to be set correctly.
>
> Works for me. Can you give an example?

Yes. I'm in the UK and on Windows (4.2.2 and 4.4.0):
>> localtime(time)
    usec =  178342
    sec =  59
    min =  32
    hour =  8
    mday =  12
    mon =  4
    year =  118
    wday =  6
    yday =  131
    isdst =  1
    gmtoff = 0
    zone = GMT Daylight Time

>> merely setting TZ is not sufficient - localtime and maybe other functions
>> seem to reference a cache that is only loaded by mktime.

> Does the following not work for you?
>     x = time ();
>     t1 = localtime (x)
>     setenv TZ UTC0
>     t2 = localtime (x)
>     setenv TZ CST6CDT
>     t3 = localtime (x)

No. On 4.2.2 and 4.4.0 on Windows, after running this I get t1==t2==t3=
    usec =  140346
    sec =  41
    min =  35
    hour =  8
    mday =  12
    mon =  4
    year =  118
    wday =  6
    yday =  131
    isdst =  1
    gmtoff = 0
    zone = GMT Daylight Time

> If not, that may be a defect in your operating system, it works for me.

Assuming you meant "a defect in the Octave build for Windows" rather
than "a defect in Windows", it would seem highly likely.

> I don't think it's within the scope of the Octave documentation to
> describe the TZ environment variable. I think it would be appropriate to
> refer to the section in the GNU libc manual.

That's one philosophy. I believe though that many Octave users are
scientists with no interest in programming. So a little more help may be
appropriate when they need to use functions that give access to operating
system services, and especially operating system services for an
operating system and programming langage they are not running and
cannot get support on. And I note that your philosophy was not applied
in section "14.2 C-style I/O functions", which documents printf etc
without reference to C manuals.

There is also the question of what setting TZ affects. For Octave on
windows (which has no concept of process-local timezone, so no
TZ or similar environment variable) it just affects the behaviour of
octave timing functions. For Octave on Unix it (I assume) changes
the effective timezone for system services also.

>> 6. Bug, or documentation error. "Leap Seconds" are mentioned as
>> ignored in text under the datenum function, but leap seconds apply to
>> many APIs and what "ignored" means is ambiguous. If it is decided that
>> a particular day will have 86401 atomic clock-defined seconds does
>> "ignored" mean the seconds counter ranges 0:86400 instead of 0:86399?
>> Or does it mean the clock is ever after out of step with the world's
>> rotation and almost all other clocks in the world? Or does it mean the
>> system on which Octave is running has slowed its timing services for a
>> few hours or days to remain in sync with the accepted world standard
>> time.
>
> It simply means that in the context of the datenum function, every day
> is exactly 86400 seconds. The following two expressions return the same
> datenum value:
>
>     datenum (2016, 12, 31, 15, 59, 60)
>     datenum (2016, 12, 31, 16, 00, 00)

So date-time arithmetic in general (not just datenum) does not reference
a table of leap seconds, and as a corollary, Octave seconds(*) are on
average bigger than TAI seconds,
https://www.timeanddate.com/time/international-atomic-time.html, to
compensate. I think it is worth saying.

(*) Yes of course Octave seconds are System seconds, and virtually all
systems will take their time from a public NTP service which will have a
strategy for dealing with the problem, e.g.
https://cloudplatform.googleblog.com/2016/11/making-every-leap-second-
count-with-our-new-public-NTP-servers.html.

Ian


Cheers... Ian

Reply | Threaded
Open this post in threaded view
|

Re: Octave date and time services

Mike Miller-4
On Sat, May 12, 2018 at 17:15:23 +0100, Ian McCallion wrote:
> I think I can best serve
> Octave by writing up the bug reports you would like written up and the
> next step should be for you to say which they are.

Ok, sounds good.

> Yes. I'm in the UK and on Windows (4.2.2 and 4.4.0):
> >> localtime(time)
>     usec =  178342
>     sec =  59
>     min =  32
>     hour =  8
>     mday =  12
>     mon =  4
>     year =  118
>     wday =  6
>     yday =  131
>     isdst =  1
>     gmtoff = 0
>     zone = GMT Daylight Time
Ok, on Windows, the gmtoff and zone fields are not implemented. The
gmtoff field will always be zero, and Octave will call tzname to
populate the zone field. This may not be the ideal way to represent that
these fields are optional and may not be valid on a particular system.

This could be documented in the Octave user manual.

We could also include documentation that these fields are extensions and
are informational only, they are not used by the mktime function.

> No. On 4.2.2 and 4.4.0 on Windows, after running this I get t1==t2==t3=
>     usec =  140346
>     sec =  41
>     min =  35
>     hour =  8
>     mday =  12
>     mon =  4
>     year =  118
>     wday =  6
>     yday =  131
>     isdst =  1
>     gmtoff = 0
>     zone = GMT Daylight Time
>
> > If not, that may be a defect in your operating system, it works for me.
>
> Assuming you meant "a defect in the Octave build for Windows" rather
> than "a defect in Windows", it would seem highly likely.
The POSIX function tzset serves the purpose of copying the TZ
environment variable into the runtime for other time functions to use.

According to the GNU libc documentation,

| It is not usually necessary for your program to call this function,
| because it is called automatically when you use the other time
| conversion functions that depend on the time zone.

It appears that the localtime function on GNU/Linux operating systems
automatically calls tzset to update the internal time zone information
from the TZ environment variable. On Windows, it seems to be that the
localtime function does not, but the mktime function does. That's what I
meant by defect in your operating system.

I think this would be appropriate to fix in Octave, by explicitly
calling tzset wherever it may be needed.

> > I don't think it's within the scope of the Octave documentation to
> > describe the TZ environment variable. I think it would be appropriate to
> > refer to the section in the GNU libc manual.
>
> That's one philosophy. I believe though that many Octave users are
> scientists with no interest in programming. So a little more help may be
> appropriate when they need to use functions that give access to operating
> system services, and especially operating system services for an
> operating system and programming langage they are not running and
> cannot get support on. And I note that your philosophy was not applied
> in section "14.2 C-style I/O functions", which documents printf etc
> without reference to C manuals.
I think it's fair to file a bug report to document, at the very least,
the existence of the TZ environment variable. How much depth that
documentation will go into can be discussed.

> There is also the question of what setting TZ affects. For Octave on
> windows (which has no concept of process-local timezone, so no
> TZ or similar environment variable) it just affects the behaviour of
> octave timing functions. For Octave on Unix it (I assume) changes
> the effective timezone for system services also.

I'm not sure I follow here. I think GNU and Windows and Unix operating
systems all support a TZ environment variable and a concept of
per-process time zone. Setting the TZ environment variable in Octave
never has an effect on the system time zone setting.

> So date-time arithmetic in general (not just datenum) does not reference
> a table of leap seconds, and as a corollary, Octave seconds(*) are on
> average bigger than TAI seconds,
> https://www.timeanddate.com/time/international-atomic-time.html, to
> compensate. I think it is worth saying.

Correct.

> (*) Yes of course Octave seconds are System seconds, and virtually all
> systems will take their time from a public NTP service which will have a
> strategy for dealing with the problem, e.g.
> https://cloudplatform.googleblog.com/2016/11/making-every-leap-second-
> count-with-our-new-public-NTP-servers.html.

Right.

Cheers,

--
mike

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Octave date and time services

Mike Miller-4
On Mon, May 14, 2018 at 11:53:57 -0700, Mike Miller wrote:
> It appears that the localtime function on GNU/Linux operating systems
> automatically calls tzset to update the internal time zone information
> from the TZ environment variable. On Windows, it seems to be that the
> localtime function does not, but the mktime function does. That's what I
> meant by defect in your operating system.

Actually, this is because Octave uses the Gnulib mktime replacement
function in the Windows build, which does explicitly call tzset.

> I think this would be appropriate to fix in Octave, by explicitly
> calling tzset wherever it may be needed.

Please do file a bug report that all low-level time functions dealing
with local time should ensure that tzset is called.

--
mike

signature.asc (849 bytes) Download Attachment