Monday, March 4, 2013
We're happy to announce the new way to file a contributor agreement: on the web at http://www.python.org/psf/contrib/contrib-form/.
Through the use of Adobe's EchoSign, we got rid of the old hand-written, print out, scan or photograph, then fax or email of your form. It was a hassle for our contributors, and a hassle for our administrators. Faxes fail, mail gets lost, and sometimes pictures or scans turn out poorly. It was time to find a more user-friendly solution, and the Foundation is happy to finally offer this electronic form.
The new form is easy to fill out right on the site, guiding you through each of the required fields such as your name, bug tracker ID, address, and initial license. If you're signing the form on behalf of an organization, there's a check box to specify this, and then you are asked near the bottom to state your title in the organization. Lastly, your signature is either generated from your typed name, or you can draw your own or upload a signature file of your own.
Once you submit the form, you'll receive an email from echosign.com to verify the email address you entered. Once you click to confirm your address, the form will be emailed to the PSF and will be recorded.
We require all contributors to CPython to have a signed form, and we hope this makes it easier for potential contributors to join up and help make Python better. It's available just in time for PyCon and the CPython sprint that will be occurring March 18 through 21 in Santa Clara, California. Join us at the sprint, sign your contributor form, and help us fix some bugs or add some features!
Tuesday, February 19, 2013
Christian Heimes announces the release of his defusedxml and defusedexpat packages to address XML-related security issues which were reported to email@example.com over the last several months. Throughout the development of the patches, the security team has coordinated with other open source projects in order to make this announcement at 1500 UTC on Tuesday February 19.
Details will follow once releases of CPython have been organized.
Note: this post will be updated with more details as they switch from being private to publicly available, including links to the public bug reports on http://bugs.python.org.
defusedxml on PyPI: https://pypi.python.org/pypi/defusedxml
defusedexpat on PyPI: https://pypi.python.org/pypi/defusedexpat
"XML vulnerabilities" on bug tracker: http://bugs.python.org/issue17239
<!DOCTYPE xmlbomb [ <!ENTITY a "1234567890" > <!ENTITY b "&a;&a;&a;&a;&a;&a;&a;&a;"> <!ENTITY c "&b;&b;&b;&b;&b;&b;&b;&b;"> <!ENTITY d "&c;&c;&c;&c;&c;&c;&c;&c;"> ]> <bomb>&d;</bomb>
<!DOCTYPE bomb [ <!ENTITY a "xxxxxxx... a couple of ten thousand chars"> ]> <bomb>&a;&a;&a;... repeat</bomb>
<!DOCTYPE external [ <!ENTITY ee SYSTEM "http://www.python.org/some.xml"> ]> <root>ⅇ</root>
- An attacker can circumvent firewalls and gain access to restricted resources as all the requests are made from an internal and trustworthy IP address, not from the outside.
- An attacker can abuse a service to attack, spy on or DoS your servers but also third party services. The attack is disguised with the IP address of the server and the attacker is able to utilize the high bandwidth of a big machine.
- An attacker can exhaust additional resources on the machine, e.g. with requests to a service that doesn't respond or responds with very large files.
- An attacker may gain knowledge, when, how often and from which IP address a XML document is accessed.
- An attacker could send mail from inside your network if the URL handler supports smtp:// URIs.
<!DOCTYPE external [ <!ENTITY ee SYSTEM "file:///PATH/TO/simple.xml"> ]> <root>ⅇ</root>
|external entity expansion (remote)||True||False (3)||False (4)||True||untested|
|external entity expansion (local file)||True||False (3)||False (4)||True||untested|
|xpath support (7)||False||False||False||False||False|
|xsl(t) support (7)||False||False||False||False||False|
|xinclude support (7)||False||True (6)||False||False||False|
- Lxml is protected against billion laughs attacks and doesn't do network lookups by default.
- libxml2 and lxml are not directly vulnerable to gzip decompression bombs but they don't protect you against them either.
- xml.etree doesn't expand entities and raises a ParserError when an entity occurs.
- minidom doesn't expand entities and simply returns the unexpanded entity verbatim.
- genshi.input of genshi 0.6 doesn't support entity expansion and raises a ParserError when an entity occurs.
- Library has (limited) XInclude support but requires an additional step to process inclusion.
- These are features but they may introduce exploitable holes
- Don't allow DTDs
- Don't expand entities
- Don't resolve externals
- Limit parse depth
- Limit total input size
- Limit parse time
- Favor a SAX or iterparse-like parser for potential large data
- Validate and properly quote arguments to XSL transformations and XPath queries
- Don't use XPath expression from untrusted sources
- Don't apply XSL transformations that come untrusted sources
- Unrestricted entity expansion induces DoS vulnerabilities in Python XML libraries (XML bomb)
- External entity expansion in Python XML libraries inflicts potential security flaws and DoS vulnerabilities
- Brett Cannon (Python Core developer)
- review and code cleanup
- Antoine Pitrou (Python Core developer)
- code review
- Aaron Patterson, Ben Murphy and Michael Koziarski (Ruby community)
- Many thanks to Aaron, Ben and Michael from the Ruby community for their report and assistance.
- Thierry Carrez (OpenStack)
- Many thanks to Thierry for his report to the Python Security Response Team on behalf of the OpenStack security team.
- Carl Meyer (Django)
- Many thanks to Carl for his report to PSRT on behalf of the Django security team.
- Daniel Veillard (libxml2)
- Many thanks to Daniel for his insight and assistance with libxml2.
- semantics GmbH (http://www.semantics.de/)
- Many thanks to my employer semantics for letting me work on the issue during working hours as part of semantics's open source initiative.
Thursday, December 20, 2012
On Raymond Hettinger’s recent thread about the memory layout of dictionaries, Barry Warsaw and Christian Heimes shared concerns about how things might look on ARM devices. Christian mentioned the Snakebite environment, run by Trent Nelson, but without any ARM machines in the environment, Trent offered to host the boxes if someone donates them.
Based on the thread’s suggestions and the low cost of the devices, the PSF authorized purchase of a PandaBoard ES, featuring a 1.2 GHz ARM Cortex A9, along with several accessories to get it running. The PSF already had a few Raspberry Pi devices on hand, which come with a 700 MHz ARMv6, so one was dispatched to Trent.
Thanks to the PSF for making the purchase, and thanks to Trent for offering to set up the machines and add them to the environment!
Monday, November 19, 2012
The survey is available at https://limesurvey.sim.vuw.ac.nz/index.php?sid=65151&lang=en.
Kevin states, “The goal of this research is to understand how a person's experience as a newcomer to a Free/Open Source Software (FOSS) community influences that person's behavior and contributions within that community.” He estimates that the survey will take around 20 minutes to complete.
Throughout the nearly three year period since January 2010, over 40 committers were added and countless others contributed patches, reviews, and bug triage. Since the creation of the Core-Mentors group in March 2011, we’ve seen many first timers come through and have their work committed and released.
We hope that the mentorship group has helped introduce contributors in a positive manner, so we’re looking forward to the results of Kevin’s studies. One of the things Kevin hopes to find is whether or not formal mentorship programs work for introducing contributors. He also looks to find answers to how much community support of newcomers matters, whether formal joining processes involving sponsorship work, and if newcomer specific tasks are the way to go.
If you have the time, please fill out the survey. Python is participating in this survey along with several other groups including Debian, KDE, Gnome, openSUSE, and OpenHatch. Perhaps we can learn a few things and create an even better experience for new contributors!
Tuesday, October 30, 2012
Join us for an effort at closing some Python bugs and feature requests. Get quick feedback on your patches and bugfixes, learn how to submit and examine patches, and have fun chatting with the Python developers and other contributors. You don’t need to know the CPython codebase or process to join, just Python programming knowledge.
If you live in Montreal, come at Caravan to meet fellow hackers and take part in a physical sprint!
Please register to let us know how many people to expect. People from around the world are should join the #python-dev IRC channel to participate in the bug day.
- Grab a copy of the Python codebase from Mercurial, following instructions in the Developer's Guide, and compile it.
- If you have a problem that isn't in the bug tracker, announce it to the IRC channel, and if it's more than five minutes' work, create a bug report for it. See the bug reporting instructions to learn how to write bug reports.
- When you choose a bug to work on, announce it to the IRC channel (e.g. "I'm working on #123456.") or on the bug report itself. This avoids accidentally duplicating work.
- Consider providing a patch that fixes the problem, or at least a simple test case that demonstrates the bug. Please see the patch submission guidelines in the Developer's Guide before submitting a patch.
- Does the bug appear to be gone in the Python development version (the Mercurial branch "default", that will become 3.4), but not the 3.2, 3.3 or 2.7 maintenance branchs? Report that, too.
- If someone else has supplied a fix, see if this fix works for² you, and add your results to the bug.
- Read the text of proposed patches and assess them for correctness and code quality. This is usually the most time-consuming step in the bug fixing process, so reading patches is very useful.
- If there's a working fix, feel free to add a note asking for the fix to get committed. The bug tracker has a lot of items in it, and it's easy for bugs to be overlooked.
- Feature requests should be classified as type 'feature request' in the bug tracker.
Monday, October 29, 2012
Over the weekend, PEP 430 was approved, which changes the default documentation displayed at http://docs.python.org. See the PEP for full details, but the jist is that we're now promoting the current Python 3 release as the default when you go to the docs home page. However, as the majority use case is still for Python 2 documentation, navigating straight to an unversioned page will present you with the current Python 2 documentation. For example, an unversioned link such as http://docs.python.org/library/zipfile will bring up the 2.7.3 documentation.
Supporting that change is a new feature that adds a version dropdown to the top of all documentation pages. Not only does this help when users are brought to a page which they don't expect, but switching between versions is a common operation as more and more projects work to add support for Python 3. Issue 8040 is where you'll find discussion on the change and its patches, with the bulk of the work completed by Yury Selivanov with some help from Georg.
This dropdown is especially handy as you peruse the documentation and come to a page that you want to view in another version. Choosing another version while on any page will load that page's other version, where the latest release of that version is chosen, e.g., 2.7 currently points to 2.7.3. So, as you browse the 2.7.3 built-ins page, choosing 3.3 in the dropdown will bring you to the 3.3.0 built-ins page.
We hope these changes enhance your experience when browsing the Python documentation!
Tuesday, August 14, 2012
Release manager Georg Brandl announced on August 12 that the second beta of CPython 3.3 was released, complete with installers for both Mac and Windows. This release represents the final feature set, and the goal is to get it in the hands of users to iron out any last issues.
Following this beta will be two release candidates, coming August 25 and September 8. The final release is slated to happen on September 22.
The "What's New in Python 3.3" document is currently being finalized by curator and long time developer Raymond Hettinger. The document already contains many of the new changes, but keep an eye out for newer versions.
Here are some of the bigger changes:
- PEP 380, syntax for delegating to a subgenerator ("yield from")
- PEP 393, flexible string representation (doing away with the distinction between "wide" and "narrow" Unicode builds)
- A C implementation of the "decimal" module, with up to 80x speedup for decimal-heavy applications
- The import system (__import__) now based on importlib by default
- The new "lzma" module with LZMA/XZ support
- PEP 397, a Python launcher for Windows
- PEP 405, virtual environment support in core
- PEP 420, namespace package support
- PEP 3151, reworking the OS and IO exception hierarchy
- PEP 3155, qualified name for classes and functions
- PEP 409, suppressing exception context
- PEP 414, explicit Unicode literals to help with porting
- PEP 418, extended platform-independent clocks in the "time" module
- PEP 412, a new key-sharing dictionary implementation that significantly saves memory for object-oriented code
- PEP 362, the function-signature object
- The new "faulthandler" module that helps diagnosing crashes
- The new "unittest.mock" module
- The new "ipaddress" module
- The "sys.implementation" attribute
- A policy framework for the email package, with a provisional (see PEP 411) policy that adds much improved unicode support for email header parsing
- A "collections.ChainMap" class for linking mappings to a single unit
- Wrappers for many more POSIX functions in the "os" and "signal" modules, as well as other useful functions such as "sendfile()"
- Hash randomization, introduced in earlier bugfix releases, is now switched on by default
In total, almost 500 API items are new or improved in Python 3.3.