Category: Uncategorized

Kernel Versions 1.x.x

As legacy code still used by the kernel provides many basic features already it will probably be pointless to call versions still using old/third-party code 0.x.x so instead they will be numbered starting around 1.7.x or 1.8.x reflecting progress towards a fully-new “2.0” kernel product planned for some time in 2026.

The 1.x.x releases will therefore partly be focused on establishing stable APIs for 2.0 including for programs to check what OS version they’re running on, and also on dampening any expectations for a perfect complete OS arriving at the first major release.

The versions will probably be marked internally as something like “SLOS 1.7.0”, the uname formatting will be finalised shortly. The main system APIs/ABIs should stabilise towards the end of the 1.x.x series (so 1.9.x should work mostly like a stable 2.0 version, but may break some compatibility with 1.7.x & 1.8.x versions).

Compiler Versions ~0.9.x

The compilers are close to usable for everyday stuff but will have more bugs in practice than the kernel code. So they are already usable but won’t really be “finished” until there’s a whole ecosystem for testing & deploying them.

If I get lazy these versions will be tied the OS version but maybe minus 1.0, so OS version 1.8.0 may match compilers versioned 0.8.0, and after 2.0/1.0 releases then things will be versioned in a more fine-grained way based on individual product features.

Other Tools Probably Tied To Kernel Version

For now the other tools like the package manager/archiving program will probably just be tied to the kernel version, so version 1.8.0 of the kernel will come with version 1.8.0 of the package manager and some “version 1.8.0” installations or packages of other tools, even if they aren’t necessarily updated for each kernel version.

This is just a practical consideration as obviously I don’t have a full team to offload managing new versions of each component to!

Kernel 2.x, Compilers 1.x

After a few proper early releases then the versioning will start to change to reflect individual updates & stability of each product. So for example the OS as a whole might eventually just have 2026 & 2027 versions while the package manager or other individual components might stay at something like version 2.1 for a while.

The programming tools will probably have simple versioning schemes easy for programmers (so probably not bumping major versions up every year, and maybe incorporating some rule like “1.2 is a stability release, 1.3 is a feature release, 1.4 is a stability release, 1.5 is a feature release”).

But that’s a matter for later, for now I will probably just tie most versioning to progress towards the “2.0” kernel rewrite.

Filesystem & Protocol Versions

The filesystem code already has an established versioning system and any other new protocols or formats will probably have individual versions too, these will be documented separately to software release versions and won’t match them 1:1.

Although the new filesystem format version is also “version 2”, the filesystem code supports multiple format versions and has it’s own options for setting or displaying the format version in use.

Platform-First Planning

The main goal is to develop a usable general-purpose operating system platform, not to finish one individual component to perfection or to achieve complete technological self-sufficiency. However, being very interdependent, the OS “release versions” should coincide with some individual components becoming more stable.

Necessary & Nice Features

For an okay-ish release version it will be necessary to finish some things:

• More documentation (especially of bugs/limitations)
• Reasonably stable kernel
• Reliable demos
• Some useful drivers/modules
• Build system improvements for “release builds”
• Downloads & backups for each release

For a perfect release version it would be nice to have:

• Fully new kernel code
• Bug-free C compiler
• Fully self-hosting codebase edit/compile/test
• Extensive manual pages
• Usable graphics/networking support
• Full packaging & versioning of each component
• Lots of new kernel features like dynamic modules

A Modest 1.x Release Plan

The “necessary” features can be thrown together very easily in a limited form if need be but working towards “nice” features should dictate the actual targets for a meaningful “point zero” release.

So I’ll revise my planning a little as I do further tests but I’ll probably drop a “first full release” version once I’ve finished off some more “nice” features.

A Possible Trick

As most of the code doesn’t currently have a versioning scheme but is already in a mostly working state, one trick I might use is to just bump up the kernel version automatically to 1.x.x based on the xx% replacement of legacy code and base the platform version on that.

This would make the initial releases smooth 1.x.x values starting around 1.7.0 or 1.8.0 while avoiding any possibility of perpetual “0.9.9 final beta 10 almost finished” type releases, while an eventual 2.0 release down the track would be the “fully new kernel code & stable API” release.

On the other hand if these milestones approach too quickly it would be pointless to call a final version 2.0 I might as well just do 0.8, 0.9, 1.0. So I’ll see how I go with some upcoming features and work from there!

Kernel Audit Milestone

Kernel audit tool now reports 67% new code meaning now less than 1/3 of the kernel is still based on legacy code.

This milestone coincides with most of the system call API now having been rewritten (but still heavily depending on some old and partly updated functions internally).

System Calls Need More Fixes

The current API still mostly replicates the old one with a few changes only where necessary, this makes testing across updates a little easier for now.

Future versions will improve details such as using 64-bit sizes/offsets/indices where applicable. Proper system call documentation will follow as I start to prepare a final “1.0” API.

Coming Up

• Build system improvements especially for dealing with different hardware configurations
• Scheduler & threading improvements
• Further work on container support & maybe some prep for hardware virtualisation
• More work towards proper hardware drivers

I’m having a lot of troubles with the local authorities and a couple of issues with things like uploading photos on the new site (works ok from my development machines but not phone…), but in good news I’m still managing to get new test hardware from China.

A couple of models of motors arrived this last week, I’ve got a surplus of boards for doing more extended testing once I can get a proper workbench set up somewhere, and I’ve already paid early for a discount on a new high performance board (the Titan) I’ll hopefully be ordering very soon. (UPDATE: Okay maybe the preorder vouchers were cancelled? Waiting to hear back from the reseller about what’s going on with that launch.)

I’ve now written some replacement code for PLIC (the interrupt controller on RISC-V) as well as some new code for toggling GPIO pins but this is left to be tested & finished another day.

A full device driver kit will come later but I figured access to GPIO pins would be the most useful driver overall! I’ve found a bit of reference information for GPIO on a couple of devices and I’ve only implemented limited support for one so far but it should be straightforward to get some GPIO going on different devices as long as I can find enough information.

I’ll need to decide on some API for GPIO access before this will be usable from user programs or from other drivers. The other important part needed to get some drivers working is some mechanism to configure devices from the device tree info, at the moment the system parses the file but doesn’t do much with the information except for detecting memory, so it will at least have to pass on some base addresses or other info to individual drivers.

Increased testing has identified a number of problems and is starting to lead to solutions. This post will go over the status of some recent changes as well as test results.

Filesystem

The filesystem update is now almost complete, with only around 400 lines of legacy code remaining to be replaced in filesystem syscall functions.

Testing revealed one major stability issue resulting in inode table exhaustion, this was a simple reference counting error and has been fixed.

Overall the filesystem code is mostly stable for simple deployments but extensions like support for large disks are still in progress. The filesystem code supports using filesystems from earlier versions, allowing for more stable & more full featured options.

Some VFS-like support also exists by way of a “drives” structure for multiplexing filesystems, which is designed to work with the container system, but this will be discussed in another post.

Compiler

Testing also revealed some problems when using the filesystem code with the new compiler backend, these have partly been worked around with rewrites but some compiler bugs are still affecting the filesystem code.

The compiler is likely to still have some known bugs at release, so will initially be a demo together with the new system (building the system with GCC will be more stable for the near future). Once features are finished more compiler testing will be enabled hopefully resulting in a very stable platform down the track.

Scheduler

A few tests have been crashing the scheduler, which sounds bad, but I have already written a new fallback scheduler implementation which should address any such stability issues while the main scheduler implementation is improved.

So as the kernel code stabilises there should actually be separate “stable” and “performance” scheduler options, and hopefully these will be usable side-by-side on the same system.

Container Security

I’ve avoided adding legacy security features like user/group support or fine-grained capabilities in favour of native support for containers. This should make for easier real-world security as the container model can easily be adapted for multi-user and multi-app security barriers, and this model fits the design of the rest of the system.

Container support half-exists in the kernel already so some things already go through the container layer, but proper management of containers & other security features won’t be fully enabled until later. Due to the system being designed around this model it should be fairly robust relative to early container implementations, with a different set of core kernel structures for each “container” and possibly eventually some address space isolation between them too (not yet implemented).

Hardware Ports

Testing on real hardware has finally commenced but has been slightly delayed by the need more more testing of core features (which I’ve been doing in an emulator), so I hope to get back to that shortly.

Initial testing has been done on the Ky X1 CPU with memory & multicore support fully or almost fully working as of the last tests but no hardware drivers for testing any real stuff (only a serial console & RAM disk). This work is expected to also be applicable for booting on other RISC-V 64 devices but I don’t know how similar other driver needs will be across devices yet.

Once I can get back to hardware work I hope to at least get GPIO drivers working on one or two boards and add more interrupt handling support so that drivers for simple robotics devices can start being implemented.

Networking & Graphics

Some work has been done on these subsystems at least in planning but probably won’t be fully enabled at release. Some demos may be possible soon, don’t hold your breath.

Kernel Audit

The kernel is currently around 64% new code. Old code continues to be replaced, however early development also swelled the original codebase a little leaving currently just under 5,000 lines of “old” code still left to replace.

The filesystem code is now almost entirely new and supports some extended format versions to start testing more heavy duty workloads. The few bits of old filesystem-related code are being steadily replaced now.

The scheduler code is partly new but leans heavily on old functions in proc.c, further scheduler improvements will probably be synchronised with the container support.

The rest of the kernel is about half new/half old with things like support for device tree blobs added recently but some other startup code & utility functions being old. Some unused or redundant code is still laying around (such as currently having two copies of some page table functions), so the code statistics are a little displaced by such things but represent the in-development codebase.

Userland & Tests

Some work has been done on new “userland” programs & ports, so testing has also been done with some almost-real-world programs but this will be covered in another post. The “libc” implementation is almost entirely my own code, it’s missing many functions but otherwise is working quite well.

The old test suite itself is still not completely working so around half the tests are still disabled, but support is improving. An additional test also exists for multithreading, which is partly stable and will probably be enabled at release but with limited functionality.

Summary

With the filesystem code relatively stable and stability of the scheduler improving I seem to be on track to deliver a stable and high performance core OS, but of course am behind schedule and need to finish many things.

Some progress has been made porting the test suite from xv6 to the new compiler backend & updated kernel.

One serious stability bug was discovered in the filesystem code, with the system running out of inodes after some number of runs of the test suite (meaning files aren’t being closed properly somewhere…), and some bugs in the compiler are also exposed by the test suite. About half of the original tests are currently enabled, with some additional bugs to investigate in the others (while some have just been disabled due to being too slow or configuration-sensitive).

Will try to return to code!

(If you can’t tell!)

I’m aware of some minor problems with the site like menus not looking good on mobile and missing contact forms, I’ll try to fix these eventually but will be prioritising other things.

In the long term hopefully I’ll be able to hire someone to maintain the site but for now I have a lot of things to balance and will probably have to accept a few annoyances with site maintenance to avoid losing too much time tweaking CSS.

If you check back in a month or so I should have the site more prepared, in the meantime I’ll try to post more updates, write some better introduction pages and ready some actual code releases. Your patience is appreciated!

Aside from coming up with ideas for “release engineering” there are a few parts of the OS which are currently under heavy development and may or may not be ready in time for the first proper releases:

PLIC & Driver Kit

The system currently only supports some ad-hoc hardware drivers for testing, but does support loading Device Tree Blob files so can be extended to support real hardware drivers with a bit more work:

• PLIC & interrupt handling code needs to be extended or rewritten to allow dynamic configuration from the DTB
• Management of modules & drivers is not fully fleshed out yet, but is likely to work similarly to other systems (e.g. “device files” are still used currently with limited functionality but could be streamlined into a more modern configuration)
• Alternative filesystems, advanced disk management and partitioning support will probably be delayed until after some releases or released with limited functionality (prioritising enhancements to the core filesystem instead to ensure it remains stable)
• Framebuffer graphics can easily be added for some (emulator) targets but will likely be more problematic on real hardware
• No work has commenced on any support for device hub technologies such as PCI & USB, support for legacy device hubs will probably be limited for a little while in favour of drivers for smaller individual devices & GPIO

Full Self-Hosting

The system is already quite stable when built with the custom compiler, but more work is needed especially on the assembler/linker and at the build system end in order for the OS to be able to fully recompile itself.

In the meantime most development is done from Linux, which is a reasonable compromise and is the same way most other operating systems are developed, but the system is fairly close to being self-hosted and this is expected to go very smoothly once the new linker rewrite is finished.

Networking Support

I’ve been looking into a few different options to provide networking support, none of these options are perfect so the final system may come with a couple of alternatives but these probably won’t be ready in initial releases:

• Writing a new TCP/IP stack – this is likely to be viable for small demos but is unlikely to result in a competitive system in the short term
• Porting an existing TCP/IP stack – this is likely to be much more practical but will probably be limited to some typical “embedded networking” feature set and perhaps a bit clunky to set up for end users
• Developing new protocols – this will likely result in a much better demonstration systems and perhaps will be more useful for fully integrated products (especially for HPC & robotics) but won’t integrate nicely with legacy tech
• Running full networking using virtualised/emulated systems – this will probably be a good compromise for full featured networking in the mid term but won’t solve short or long term integration problems

For the time being, some legacy internet APIs including kqueue/kevent functions are already exposed in the C library but aren’t implemented in the OS, allowing some networked programs to be built or prototyped using legacy APIs but they won’t run on the new kernel yet.

Documentation & End User Utilities

Documentation is currently limited and no man (manual page) program is included. This will be addressed as individual components stabilise, in the meantime you will need to check instructions printed by the individual programs or written in the source code.

Some other utilities needed by users but not critical for development will also have been overlooked at this stage, but more programs can easily be ported later.