Sunday, December 23rd, 2012

A Python-based Operating System

Think of how Android uses Java, but instead of Java, we replace it with Python!

I have been pondering the idea of building a Python-based Operating System on top of the Linux kernel, much like how Android uses Java. However, I wanted to take it a step further and build most of the system components using Python, this way most of the Operating System can be modified by anybody who understands Python code. This can allow anybody with Python experience to extend and fully customize the entire operating system. Since Python allows modules to be built using C, this can allow specific system components to benefit from speed where needed, such as a graphical interface. Using C interfaces also allows every part of the Linux kernel to be easily exposed to Python when attempting to build such an Operating System.

A large benefit of such an operating system is application portability, at least applications which are made in pure Python. The entire Operating System will run anywhere you can compile the Linux Kernel and Python into, with less effort than say porting all of Linux over to a different chipset. Python apps don't need to be re-compiled for the target CPU, and if most of the Operating System is made in Python, porting is made a cinch.

If such a project where to begin, I would imagine it will first be more of a research Operating System with no absolute use. It will allow Python developers to run their code very close to the bare metal of a CPU with minimal effort, and for embedded developers this type of project might actually speed up existing development processes.

What do you think? Would you use such an Operating System? If so, what would you use it for?

To discuss and develop this idea further head over to Uarica!

Comment #1: Posted 1 year, 7 months ago by Jay Aurabind

Its novel idea. But would it be worth the pain? I mean, would it give you enough performance? Python being interpreted, and not very efficient as C or C++, wouldn't it be a problem for these low power fellows ?

Comment #2: Posted 1 year, 7 months ago by Kevin Veroneau

Specific OS components will be compiled in C of course, since Python supports this using many different methods. Python shouldn't be much slower or faster than a Java VM, and Java works nicely on embedded devices. The main point of using Python over Java is licensing. After seeing what Oracle is doing to Google in Court over Android and Java, developing a Python solution would prevent any such issues and innovation can continue. The reason for not just using a strictly C system, is that Python is much more dynamic at runtime and with computers ever so changing, using a static language at the Operating System's core seems rather limiting.

Comment #3: Posted 1 year, 7 months ago by Fred Blais

I would be glad to offer my help for this project. I designed a lot of embedded systems on different microcontroller architectures with asm and C. With the lower cost of high performance cpu and high density memory, we're seeing a lot of designs that use Linux distros (pi and the like) and even Android. Personally, I don't like Android internals that much and find all these Linux distros a bit heavy. In fact, hacking into the Linux kernel is quite a difficult task. I suppose that a Python OS would be a lot more easier to adapt to your needs. I would use a Python OS for home automation, intelligent sensor networks and robotics.

Comment #4: Posted 1 year, 7 months ago by Python OS

Python OS Project Alpha

Comment #5: Posted 1 year, 7 months ago by Don O'Donnell

Don't know much about OS internals, but wouldn't the lack of true
multi-tasking in CPython due to the GIL, be hindrance in writing
an OS?

Comment #6: Posted 1 year, 7 months ago by Kevin Veroneau

Don, the Linux kernel will be doing the multitasking. Python will not be kernel, or do any really low level stuff. It will be an OS which is mainly Python-based running on top of the Linux kernel. Think Android.

Comment #7: Posted 1 year, 7 months ago by Kevin Veroneau

Thank you for the offer Fred. I can see your use-cases being used by other people as well. I was originally thinking of bootstrapping the Linux kernel directly into Python using the kernel's init= parameter, this way a Python application can take over as INIT right off the bat and begin using the immediately available services moments after power-on. I know bash scripts can take over as init, as seen on many Linux boot floppies, so I think Python should also be-able to take on the task of being INIT as well. It will need to handle the setting up of various Linux devices and services, such as networking and perhaps udev(until there's a way for Python to control udev entirely), or just go with a static /dev tree.

Comment #8: Posted 1 year, 7 months ago by ag alex

I do not understand. Today one can write an arbitrary Linux application entirely in Python without any difficulties. And such application will be portable. What's the problem?

Comment #9: Posted 1 year, 7 months ago by mangecoeur

I've often thought it would be cool to hack on a pure python environment too! In fact the Raspberry Pi project are pushing python as their main learning language, maybe it would make sense to extend that idea to a fully pure python OS for education where people could easily change any part.
In fact i even think i know a good first step: a pip based package and update manager - essentially something that could take care of regularly updating and managing dependancies in virtualenvs so you always have a clean and up to date "OS environement". I imagine that virtualenvs (blissfully a stdlib package since 3.3) would be heavily used in such an OS for sandboxing applications...

Comment #10: Posted 1 year, 7 months ago by orcimorcim

Good idea. If you want me i join us in your project. I ve good skill in pygtk and into system embedded.

Comment #11: Posted 1 year, 7 months ago by Jordan Dimov

I like the idea a lot and would definitely use it. I don't think that any of the technical "limitations" mentioned in the previous comments are real limitations at all. Have you considered putting up a Kickstarter campaign for this project?

Comment #12: Posted 1 year, 7 months ago by João Santos

You already have a significant portion of it in Sugar http://en.wikipedia.org/wiki/Sugar_%28desktop_environment%29

Comment #13: Posted 1 year, 7 months ago by ZeD

If I remember correctly there was Pardus, a python-powered Linux distro...

Comment #14: Posted 1 year, 7 months ago by John

I seem to remember that the One Laptop Per Child project did this, or at least said that they intended to do this.

Comment #15: Posted 1 year, 7 months ago by Grant Rettke

Definitely that is a great and fun idea. Be sure to check out previous research projects so you don't have to reinvent the wheel and can avoid some of the common pitfalls.

Comment #16: Posted 1 year, 7 months ago by Shokry

That would be a great thing, Python is my language of choice and I've always been interesten in OS internals, I've read a few books but I would love to see how a real project would be, and perfecctly in Python.
I like the idea a lot, it's like a dream for me and I would like to participate or at least have an eye on what's going on.
I would use it for a smartphone.

Comment #17: Posted 1 year, 7 months ago by Anonymous

Pretty interesting this idea. Heavy, but interesting.

You can learn more about how Android do it in http://elinux.org/Android_Booting#Bootloader

Comment #18: Posted 1 year, 7 months ago by Kevin Veroneau

With so much positive input, it's hard not to start such a project... Within the next few days I will create a repository on my BitBucket account to host the initial source code I will be working on and to host various ideas. I will go through a few different ideas I have and see which one seems the most probable to implement. Since the invention of such tools like VirtualBox and User-Mode-Linux, I can quickly develop testing prototypes to test out different theories on a Python-based OS built around the Linux Kernel.

Comment #19: Posted 1 year, 7 months ago by Tylor Steinberger

I love the idea of a python os. It would be awesome to have a mechanism for staying 'always up to date.' I would use this if it were able to do all the things i need it to. Internet, movies, and of course hack on some python code. Id be extremely interested if it were able to run on mobile devices with touchscreens and would be more than willing to submit code to it. It would be awesome to have a android equivalent where python apps were actually first class citizens.

Comment #20: Posted 1 year, 7 months ago by Todd

Wouldn't this basically be the same thing as replacing shell with python or ipython?

Comment #21: Posted 1 year, 7 months ago by Winfried Maus

The only problem that I see with it is performance-related. However, as Bruce Eckel once said, this can be solved by throwing hardware at it.
On the positive side of things, this somehow reminds me of the good old Sinclair ZX81/Spectrum days - when the BASIC interpreter basically was the operating system. I like that approach. And I like the fact that a powerful Linux kernel would be lying directly underneath the Python language.
And to answer your question: Yes, I would use such an operating system. And I would use it at work. I'm in commercial satellite communications and I work at a Teleport where we host a lot of self-developed monitoring software and customized Linux servers. An OS like the one you're thinking of could remove unneeded complexity and abstraction layers from our setup. I like that idea. :)

Comment #22: Posted 1 year, 7 months ago by berdario

You might want to study about what Symbolics and other companies did before

http://en.wikipedia.org/wiki/Genera_(operating_system)
http://www.loper-os.org/?p=932

an OS with even its driver, network stack (and even its scheduler) written in a high level and dynamically typed programming language isn't a new idea (though still fascinating and really interesting)

here're some instruction to get an OpenGenera VM running (it's from 2007, so you might have to tweak things a bit)

http://www.advogato.org/person/johnw/diary/12.html

Comment #23: Posted 1 year, 7 months ago by eflauzo

This is a brilliant idea but it I can see what a giantic work need to be done, to build all of the essential components of modern OS. The real benefit is that Python is language for humans, it will be very very easy for a lot of people to contribute as Python code ideologically is writen/structured in only one way . And I wouldn't bother about compilation and optimization, I'm sure soon we will see hardware components that will run python bytecode.
I am willing to contribute to such project

Comment #24: Posted 1 year, 6 months ago by Fred

Hi Kevin, are you still interested with the project? Have you created the repo?

Comment #25: Posted 1 year, 6 months ago by Rohit

Hey looking forward to your repository so we can start reading and writing new patches :)
May be we should make a google group or something so we can get more traction going

Comment #26: Posted 1 year, 6 months ago by Kevin Veroneau

I've been a little busy lately, sorry everyone. A Google Group does sound like a good idea, this will enable everyone to provide input on the project before it begins. A repo will definitely be created soon. Does anyone object to using BitBucket and Mercurial? The first component I plan on coding is a replacement for the Linux INIT, so that Python is bootstrapped directly from the Linux kernel and then goes on to control the system. At first I will use pre-compiled Python from a distro, along with a pre-compiled kernel to keep the initial build simple.

About Me

My Photo
Names Kevin, hugely into UNIX technologies, not just Linux. I've dabbled with the demons, played with the Sun, and now with the Penguins.




Kevin Veroneau Consulting Services
Do you require the services of a Django contractor? Do you need both a website and hosting services? Perhaps I can help.

This Month

If you like what you read, please consider donating to help with hosting costs, and to fund future books to review.

Python Powered | © 2012-2013 Kevin Veroneau