This seems to be a bug in 4.2.1, but only when using the CLI. There
are several possible workarounds. First, start the octave GUI
instead of the CLI. Also, the GUI has its own preference system
which restores the last working directory by default. Make sure
that in Edit->Preferences->General->Octave Startup you are
not restoring the previous working directory.
If you want to use the CLI and have cd commands in .octaverc files
work then try cloning the most recent sources on the development
branch from Octave's Mercurial repo. There are instructions on
building from Mercurial sources on the Octave web site. I tested
this path as well and it works for me.