BBC’s micro:bit project open-sourced today!

As you may recall, there have been a couple of previous posts to this blog about the BBC’s micro:bit project  (also see PSF newsblog).

The micro:bit is a small, stripped-down, wearable computer (similar to a Raspberry Pi) and based on an nRF51 chip. The educational project, part of a larger UK program called Make it Digital, is designed to inspire children to become digital creators by giving away 1 million micro:bits to all 11 year-old UK schoolchildren this spring.

Today I heard some exciting news about the project from our good friend, Nicholas Tollervey, which I’m happy to pass along here. 

According to Nicholas: 

From the beginning the BBC have said that Python would be one of the possible languages that the device can be programmed in. The PSF is one of around twenty organisations in the project partnership. Since it’s a microcontroller, and the aim is to run Python… well, there’s a pretty obvious answer when trying to combine those two things. 

We’re incredibly pleased to announce that MicroPython runs on the BBC’s micro:bit. Furthermore, all the work done so far is being open-sourced today and the repository can be found here: https://github.com/bbcmicrobit/micropython

Right now only the code related to the MicroPython port is released. When the device is delivered, all the resources needed to recreate the entire project are to be released under an open license. The laudable intention is to provide an unencumbered legacy so others can build upon and adapt the work of the partnership that has created this device. 

To read more details and learn the story of how MicroPython came to be on the micro:bit, check out Nicholas Tollervey’s blog post found here: http://ntoll.org/article/story-micropython-on-microbit

Finally, there is much to be done. The project needs help from people with skill and experience developing for such devices. Could you contribute something to a project that will touch the lives of 1 million children and leave an open legacy that anyone could re-use? If so then please read the above-linked post and head on over to the code repository.

I hope that many of you will take Nicholas up on his request to contribute to this worthwhile project.

I would love to hear from readers. Please send feedback, comments, or blog ideas to me at msushi@gnosis.cx.

Read the Docs: growing with a little help from its friends at the PSF (and elsewhere)

Today's post, like the previous one, features a development project that the PSF has been delighted to fund once again this year.

On April 28, 2015, the PSF Board unanimously approved the following resolution:

RESOLVED, that the Python Software Foundation grant  $8,000 to Read the Docs, Inc. for developmental work.

What is RTD?

Looking for somewhere to host your open source project’s documentation in a way that will make it readily available, easy to find, fully searchable for your users, and exportable in PDF format, while at the same time offering you ease of use and the ability to add content as your project develops? Then, you’ll want to check out Read the Docs, the world’s largest documentation website for open source projects. 

Read the Docs

… hosts documentation, making it fully searchable and easy to find. You can import your docs using any major version control system, including Mercurial, Git, Subversion, and Bazaar. We support webhooks so your docs get built when you commit code. There’s also support for versioning so you can build docs from tags and branches of your code in your repository.

RTD’s History

RTD was created in 2010 by Eric Holscher, Charles Leifer, and Bobby Grace for the 2010 Django Dash. Eric tells the interesting story at Djangocon. A Django Dash is a coding contest that allows 48 hours for development and implementation of a project. Eric and his team considered what to do and decided that, since current documentation hosting was less than satisfactory, they could be of most help to the community by creating a web-based doc hosting solution. They agreed that Sphinx was the best document tool for Python, so they went with that.

According to Eric, 2011 was the year that saw RTD go … from a hobby project, into something projects depended on. At that point, they were hosting documentation for Celery, Fabric, Nose, py.test, Virtualenv, Pip, Django CMS, Django, Grapelli/Floppyforms/Sentry, mod_wsgi. Currently, they are hosting what Eric describes as a decent part of the Python ecosystem, including SQL Academy, Pyramid, Requests, Minecraft Overviewer, and many others. They have over 50 contributors, 7500 users, and get over 15,000,000 pageviews a month. The code for RTD is on GitHub and its documentation can be found on the site. Rackspace provides free hosting. A full list of features is available on the site.

Photo Credit: Aaron Hockley, October 2014 

Creative Commons license 2.00

Use of PSF Grant

The PSF award was part of a fundraising drive that opened at PyCon 2015 and brought in $24,000 USD from 157 contributions since then (see the RTD Blog). Corporate sponsors included Twilio, Sentry, DreamHost, and Lincoln Loop; with service sponsorships from Elastic Search, MaxCDN, and Gandi.

This funding will support RTD for 3 months of development work on the path toward sustainability as an open source project. More specifically, the funds will allow RTD to hire 2 part-time paid positions: Community Developer and Operations Developer (see RTD Blogpost for details and how to apply).

Furthermore, RTD intends to document its use of PSF grant money;  how development time is spent and how funds are allocated will be posted on RTD’s public Trello board.

If you’d like to help, you can contribute to RTD at Gratipay and you can follow them on Twitter.

I would love to hear from readers. Please send feedback, comments, or blog ideas to me at msushi@gnosis.cx.

PSF funds development: Armin Rigo's CFFI 1.0

In looking back over the PSF newsblog posts, it appears that most of the PSF funded projects I’ve written about were conferences, workshops, and education/outreach efforts. These are, of course, truly important. However, it’s also important to get the word out about several development projects that the PSF has sponsored in 2015. One such project is Armin Rigo’s work on CFFI 1.0. 

RESOLVED, that the Python Software Foundation grant Armin Rigo $2500 towards cffi development aimed at making cffi generated extension modules importable without runtime dependencies on an extension module build toolchain.

CFFI or C Foreign Function Interface for Python provides a way to call compiled C code, i.e., external C libraries, from Python using interface declarations written in C. This eliminates the need to use a programming language other than C and Python. At the same time, CFFI minimizes the amount of C code that needs to be written, so it really is a boon for Python developers. It works with Python 2.6 and up and with PyPy 2x and 3x. See CFFI Documentation.

CFFI has already had approximately 7 million downloads, so it is clearly of use, but its creator, Armin Rigo (who is also one of the creators of PyPy) saw room for improvement. Specifically, according to Armin, there were two main problems:

1. The difficulties of installation [which] can be seen from outside by looking at various workarounds and 3rd-party documentation that have grown into existence. For example, the setup.py of projects like cryptography, PyNaCl and bcrypt deploys workarounds that are explicitly documented inhttps://caremad.io/2014/11/distributing-a-cffi-project/.
2. The time taken at import is excessive in some cases. For example, importing pygame-cffi on a Raspberry Pi ARM board takes on the order of 10 to 20 seconds (and this is the fast case where the compiler doesn’t need to be invoked any more)

Due to the PSF grant, Armin was able to fix both problems. The 1.0 version, now in beta, is available at CFFI 1.0.

Our thanks to the amazing Armin Rigo for this very welcome tool!

Stay tuned for my next post about the PSF’s recent award of $8,000 USD for Read the Docs. 

I would love to hear from readers. Please send feedback, comments, or blog ideas to me at msushi@gnosis.cx.

PSF Python Job Board relaunched !

Announcing: the brand new PSF Python Job Board

We are happy to announce that we have successfully relaunched the PSF Python Job Board.

After almost one year of development and lots of work by our volunteers and contractors, we are now live with the new Python job board system.

New modern system

The new system is fully integrated into the python.org website. Job submitters can create an account on the system, log in and directly submit their job posting for approval by the PSF Job Board Team.

The team can then review the postings, check them against our submission criteria, possibly fixing some formatting, and then approve or reject the postings directly through a web interface.

No more sending dozens of emails back and forth to get the job template fixed and adding jobs to the website by hand.

Thank you to our volunteers

The new system was a major effort for all of us and I'd like to say thank you from the PSF to everyone who helped make this happen (in alphabetical order):

Reviewers:

• Simon Hayward
• Melanie Jutras
• Marc-Andre Lemburg
• Giles Thomas

Developers:

• Wiktor Bachnik
• James Bennett
• Jacob Burch
• Jon Clements
• Gil Gonçalves
• Simon Hayward
• Sarah Kuchinsky
• Marc-Andre Lemburg
• Berker Peksag
• Benjamin Peterson
• Frank Wiles

Plus everyone I forgot in this list (sorry; mail me and I'll have you added).

We'd also like to thank to Martin Thomas and Chris Withers, who each ran the Python Job Board for several years by email before the job board team was set up.

Job submitters

If you want to submit a job, please visit the how-to page which describes the process.

Submissions are free, but we'd appreciate a thank you in form of a donation to the PSF.

Job seekers

You can click through the jobs on the jobs listing or subscribe to the RSS feed we have for the listings.

Please note that we do not post CVs on the site. You will have to contact the companies directly.

Good luck with finding a new job !

Brand new bugs for free

As with every new system, there are still some bugs left. If you find something, please report it on the Github issue tracker.

More information

More information on the PSF Python Job Board and the relaunch project is available on our project page:

• PSF Python Job Board

If you have questions, please write to jobs@python.org.

Enjoy,
--
Marc-Andre Lemburg
Director, Python Software Foundation

Introducing Electronic Contributor Agreements

We're happy to announce the new way to file a contributor agreement: on the web at http://www.python.org/psf/contrib/contrib-form/.

Through the use of Adobe's EchoSign, we got rid of the old hand-written, print out, scan or photograph, then fax or email of your form. It was a hassle for our contributors, and a hassle for our administrators. Faxes fail, mail gets lost, and sometimes pictures or scans turn out poorly. It was time to find a more user-friendly solution, and the Foundation is happy to finally offer this electronic form.

The new form is easy to fill out right on the site, guiding you through each of the required fields such as your name, bug tracker ID, address, and initial license. If you're signing the form on behalf of an organization, there's a check box to specify this, and then you are asked near the bottom to state your title in the organization. Lastly, your signature is either generated from your typed name, or you can draw your own or upload a signature file of your own.

Once you submit the form, you'll receive an email from echosign.com to verify the email address you entered. Once you click to confirm your address, the form will be emailed to the PSF and will be recorded.

We require all contributors to CPython to have a signed form, and we hope this makes it easier for potential contributors to join up and help make Python better. It's available just in time for PyCon and the CPython sprint that will be occurring March 18 through 21 in Santa Clara, California. Join us at the sprint, sign your contributor form, and help us fix some bugs or add some features!

Python-Dev Launches Python Insider Blog

The Python development team has launched a new blog.

Python Insider is the official blog of the Python core development team. It will provide a way for people who don't follow the mailing list to get an overview of topics discussed there, and especially to learn about changes in store for Python.

The announcement includes details about how to subscribe to the blog through its RSS feed, email, and Twitter.

Python Language Summit EuroPython 2010

This summary was written by Tim Golden.

Before the main events of EuroPython 2010 a Python Language Summit took place at the Conference venue in Birmingham. Present were (in the order they sat around the table):

• Brett Cannon
• Guido van Rossum
• Holger Krekel
• Amaury Forgeot D'Arc
• Georg Brandl
• Péter Szabó
• Ezio Melotti
• Michael Foord
• Mark Dickinson
• Martin von Loewis
• Ronald Oussoren
• Tim Golden
• Marc-André Lemburg
• Richard Jones

Implementation Status Reports

Michael initiated a round-up of current and prospective Python versions for various implementations. CPython and IronPython have both just released v2.7 with IronPython offering some Python 3 compatibility via a command-line switch. The recent/current migration of Numpy and SciPy to Python 3 should give a boost to uptake.

Amaury confirmed that PyPy currently supports 2.5.2 but is looking to target 2.7. The PyPy guys also announced a C API bridging layer which should enable a range of Python extension modules to work directly with PyPy. This is only a stepping stone by way of broadening support.

Brett suggested that the Unladen Swallow merge to trunk was waiting for some work to complete on the JIT compiler and Georg, as release manager for 3.2, confirmed that Unladen Swallow would not be merged before 3.3.

email Module

The email module needs some work in Python 3. David Murray has been given some money by the PSF but needs more from other sources to complete the work. This is hampered by the legalities around commercial organisations making donations to not-for-profits when those donations are earmarked. Various suggestions were put forward with no-one sure of the legal issues. Guido suggested that we should move forward rather than stall for want of legal advice.

WSGI Update

A broad discussion arose concerning the issues debated on web-sig concerning the WSGI protocol and the bytes vs string issues. Marc-André brought up the cgi module which has similar issues under Python 3 and other examples were given, including ftplib, urllib, and some os functions.

Various solutions were put forward including a hybrid bytes-with-encoding object. This proposal was widely unpopular, but two proposals met with broad approval: that certain stdlib functions might be polymorphic, returning the type of their input as output; and that the encoding string should include its error-handling. An example of the first would be that os.getenv("HOME") would return "/home/tjg" while os.getenv(b"HOME") would return b"/home/tjg". An example of the latter would be "utf8:strict". Something of the sort already works for PYTHONIOENCODING.

The issue of a __format__ equivalent for bytes was also raised as was the idea of object methods to render an object as string or bytes, which could be used in the polymorphic functions above.

Stable Application Binary Interface

Martin spoke about the state of the stable ABI PEP (PEP-384), indicating that he was targeting 3.2. This work would reduce the need to recompile extension modules separately on Windows for every version of Python -- something especially pertinent when code has been orphaned but is still useful.

The versioned .so files PEP (PEP-3149) being worked out by Barry Warsaw overlaps with PEP 384, and would only be useful for extensions which don't target the stable ABI.

Garbage Collection

A messy discussion turned on the question of garbage collection of module objects, and the order in which finalisers are called if at all, especially when reference cycles exist. Marc-André was proposing a __cleanup__ magic function for Python modules, which would enable the implementer to define the order in which resources are released / closed down. This is quite a subtle area and raised the issue of unfinalised objects in a reference cycle whose memory has been freed out from under them but which still exist. Martin described the Java approach where finalisers are called once and then flagged so they are not called again, even if their object is resurrected. This sounded like a useful approach for Python, but would break code which expected to be able to resurrect an object during its __del__ method, which is not expected to account for much code.

Guido pointed out that no-one can be expected to hold enough of the complexities of this area of Python's implementation in their head, and that an implementation of some sort would need to be written so that the corner-cases could emerge.

Mac OS X

Ronald described the issues around the version and architecture differences on Mac OS X and especially around Tkinter (and therefore IDLE). It was agreed that two installers could be provided: one targeting OS 10.3 on 32-bit Intel/PPC; the other targeting 10.6 on 32 and 64-bit Intel. This latter would then be able to use the system's Tk 8.5. The 10.6 binary would also work for 10.5, which would be indicated in the install docs.

Mercurial Migration

The Mercurial migration should move forward once Dirkjan has finished work on his thesis. Martin insisted that a for-real repository would have to be set up so that people can really see how it would work. An outstanding issue in hg-svn prevents the Python history from being imported, but it should be fixable. Martin & Tim brought up the issue of externals, which the buildbots use on Windows to bring in and build slightly patched versions of external libraries such as OpenSSL and sqlite3.

Brett confirmed that he would like to see the stdlib broken out into its own repository which could then be shared between the different Python implementations.

Python Package Index

A discussion on the Cheeseshop / Package Index highlighted the fact that the packaging infrastructure has become increasingly important, especially since setuptools, buildout, and pip all download from it. Richard produced graphs showing the increase in package downloads over time, and attributed the recent slight tail-off to the fact that the tool-chains are now becoming more canny with respect to caching and mirroring.

Martin & Richard confirmed that mirrors are now in place, and Marc-André confirmed that he would be putting together a proposal to have PyPI hosted in the cloud. Guido pointed out that if an AppEngine implementation were desirable, he was sure that AppEngine team would support it with resources as needed. Martin didn't feel that there was a problem with loading on the box in question; it's the uptime that's behind people's concern as it's now so essential to installing and deploying Python applications.

Several people outlined the recent heated discussion over the addition of a checkbox to the PyPI user-registration pages. Tarek has already undertaken to patch PyPI to move the checkbox back one step, allowing existing distutils users to register from the command line. At the same time, Brett advised removing that functionality from distutils2 as signing up on a web page is no great hardship.

PSF Sponsored Sprints

At their May 10 meeting, the PSF Board create a new Sprint Committee to organize a series of sprints for working on Python. Jesse Noller is the committee chair.

This new "focused sprint" initiative is part of a concerted effort to attract more contributors to Python's development team and to speed the adoption of Python 3. The PSF has agreed to provide funding to groups interested in organizing sprints related to:

• Python core tasks such as bug triage and evaluating patches
• Python core documentation work
• Porting existing third-party libraries to run under Python 3
• Enhancing http://python.org/ with new content, organization, or design work
• Development of PyPI

Over time, other projects may be added as the committee identifies them. Complete instructions for proposing a sprint for your group will be posted once they are worked out.

In the mean time, the Sprint Committee needs volunteers to help launch this project. The first steps will be to set up some communication channels, and then start developing several guides for new contributors. If you want to help out, contact Jesse Noller via sprints@python.org.

See the announcement on Jesse's blog for more details.

Have You Signed a Python Developer Contributor Agreement Yet?

A key aspect of the Python Software Foundation's mission is managing the intellectual property for the Python language. This includes the source code for the C implementation of the Python interpreter, the Python standard library, and any related tools and documentation. In order to share the source code and documentation with users of Python, the PSF needs to hold a valid license that gives us permission to do so. The licensing requirement applies to every contribution, so the PSF board is contacting all contributors to make sure they have filled out a contributor agreement.

In June 2009, Andrew Kuchling analyzed the svn repository for CPython and compared the actual contributor names with the list of people who had signed an agreement. 39% of the lines in the Python 2.7 portion of the tree were committed by developers without signed agreements. Around 31% of the lines for Python 3.1 came from unsigned contributors. Through the efforts of Andrew and other PSF members, those statistics improved to 8.5% for Python 2.7 and 9.9% for Python 3.1 by February of 2010.

Our goal is 100% coverage, so if you have contributed source code or documentation, either directly or via a patch in the issue tracker, please make sure you have submitted a form. If you are unsure of your status, you can check your issue tracker account by logging in and then clicking the "Your Details" link in the left sidebar. The account details page includes a "Contributor Form Received" field, which will show the date the form was filed. If the field says no agreement was received, please take a few minutes to fill out the contributor agreement and send it in. If you think you have already signed an agreement but the tracker does not show it, contact psf@python.org so we can check our records.

Help Migrating to Python 3.0

To assist those wanting to migrate their code to Python 3.0 the developers have just announced the creation of the Python-porting mailing list. Signing up for the list is a good way to tap in to the advice of the core developers, so it's probably a good list even if you only want to lurk to begin with.

Remember that it may take a while for all your project's dependencies to migrate to the new environment, but it's available now, so fill your boots!