Actions sur le document
GitLaw: How The Law Factory turns the French parliamentary process into 300 version-controlled Open Data visualizations
Regards Citoyens - Roux, 2/07/2014
Nous republions ici cet article rédigé en anglais et publié sur les sites de la Sunlight Foundation, de l’Open Knowledge Foudation et de la coalition Opening Parliament expliquant le fonctionnement, les sources et les enjeux de notre projet LaFabriqueDeLaLoi.fr.
Law is Code!
Over the last few years, a number of people have explored the idea of inverting Lawrence Lessig’s metaphor “code is law”, looking at the evolution of laws through the lens of coding tools. The parliamentary process is indeed so similar to a collaborative software development workflow that it is only natural to try and use a version control tool such as git to track individual legislative changes.
The analogy between both processes is deep: in each case, there is a group of people collaborating on a textual artifact (bill or program source code), proposing changes (amendments or patches), adopting or rejecting them (through votes or pull requests), and iterating until a stable, public version is made available (by promulgation or release). This new paradigm to think about legislation paves the way for new, innovative approaches of law-tracking. Some exciting work has already been made, most notably in Germany: the BundesGit project invites citizens to propose their own legal modifications as “pull requests”, and Gregor Aisch produced an unprecedented visualization of modifications to one law over 40 years of amendments.
Initiated in 2011, the Law Factory project worked on the French legislative process to answer a simple question: does the Parliament actually write the law, or are MPs only validating the executive’s drafts, as most people commonly assume? A collaboration between Regards Citoyens, an NGO that has monitored the French Parliament’s work through its project NosDéputés.fr since 2009, and two research laboratories at Sciences Po Paris, the médialab and the Centre d’études européennes, the project also sought support from all over the world.
Two international conferences, in June 2012 and May 2014, gathered in Paris activists, NGOs, researchers, public servants and journalists, in order to share projects and ideas from a wide range of expertise. In June 2013, a two-day DesignCamp with the italian info-designers of Density Design and the portuguese hacktivists of Manufactura Independente led to a collaboration with Density Design to forge innovative ways to represent and explore bills throughout the legislative process.
The Law Factory: browsing through 290 adopted bills
After three years, TheLawFactory.fr was finally released on May 28, 2014 as a free software web application, combining all available information on 290 bills promulgated since 2010. All of the text of these bills and their amendments, as well as contextual documents such as debate transcripts, are redistributed as open data, published as version-controlled text into git repositories, and made accessible through four interactive tools that enable users – researchers, journalists, lobbyists, citizens, legislators and legislative staff – to browse the legislative process under various levels of zoom.
Similar to a Gantt chart, the first visualization proposes to navigate through time and discover within the legislative agenda which bills were discussed, when, within which chamber and for how long. The display can be switched from the top menu to other views: a comparative one to compare the global time taken to study each bill, and a quantitative one to consider only the times when the text was actually being discussed and not just sitting in between the two chambers. The menu also contains filters to display only the most amended bills or those that took the most time to consider. Another feature allows the user to select a theme or legislative year.
Clicking on a bill provides (on the right) a small set of metrics offering a first estimate of a bill’s controversiality, including measures of how much the actual text grew and changed during the whole process, how many amendments were proposed and adopted and how many words were spoken during the debates. Contrary to popular belief, first analyses reveal that the French Parliament impacts the law writing process significantly: 74 % of the amended texts studied were modified by at least 50 %, and 61 % increased in length by at least 50 %. Only a handful of texts – highly controversial ones – reveal a decrease in volume by the end of the parliamentary process. Clicking on the button « Explorer les articles » allows the user to study the legislative process of each bill individually.
Studying each bill’s changes individually
All of the changes measured can be further explored in a second module for each bill. Each step of a bill’s legislative process is displayed as a column: the first draft proposal from the government or an MP on the left, followed by each version successively adopted in committee and plenary (by the Senate and the National Assembly) during a first reading, sometimes a conciliation committee, and quite often a few more readings. In each column, the text is split into articles or sections grouping multiple articles. Each article at each step is represented as a box with a proportional height to the actual length in characters of its text.
Switching the display view into a “compact mode” reveals how much the whole text actually grew at each step. New articles are marked in green, while those that have been removed are marked in red. All other aticles are shaded in grey depending on how much the text of their alineas was modified during that step so that any highly rewritten article can be quickly identified. Clicking on an article gives access on the right to the full text of the article at the step, and optionnally display the differences with the previous version as within developers usual code-diff tools.
Exploring the debates and amendments of the parliamentary process
When the text of a bill was modified at a step, a third tool is made accessible to explore all the related amendments via a folder icon in the column header. Each amendment is represented as a small square with the color of its originating political party and an ideogram revealing its resulting status: adopted, rejected or left out. Here again, clicking on an amendment reveals (on the right) the complete amendment text, its author(s) and an explanation of the amendment.
Switching the view into a “grouped mode” and ordering the amendments per political party gives a visual estimate of the origin of all adopted modifications. This can also help visualize parliamentary obstruction, for example, when a political group purposely floods the debate to slow it down with hundreds of amendments destined to fail.
A last visualization, accessible via a discussion icon in the column header, proposes to get to the root of the bill changes: the actual discussions between the members of the parliament at a selected step. Presidents, rapporteurs, government members and the different parliamentary party groups share the speaking time differently during the successive parts of the debate, beginning with a general discussion, followed with a focus on each article and related amendments individually. Throughout these steps, each group of speakers is represented as a stream graph, each step being a box with a size proportional to the number of words spoken per subject.
This visualization aids the user to identify highly debated articles and evaluate the evolving position of each political party on a text. Once again, clicking on a box displays on the right the detailed list of the speakers, with links to the actual minutes of the debate as republished oat NosDéputés.fr and NosSénateurs.fr, offering an unprecedented way to trace the discussions related to specific modifications of the law in just a few clicks.
How does it work?
Like with most projects handling parliamentary data, at least two thirds of the development time had to be spent not on building the visual exploration tools, but on collecting, assembling, cleaning and processing the data. The French Senate has made great efforts towards open data over the past couple years and recently started distributing complete dumps of some of their databases on a daily basis. On the other hand, the National Assembly remains hermetic to any sign of openness, most probably due to its members’ irrational fears of already existing activity rankings by the press…
Ultimately, the Senate’s efforts were only marginally useful for this initiative. Data related to amendments and debates was already preprocessed and waiting to be reused within NosDéputés.fr and NosSénateurs.fr’s databases, only requiring some adjustments and enrichments to their APIs in order to provide direct access to a specific bill’s debates, which can now also benefit other interested users.
But the missing data required here was the version-controlled legislative one: the actual text of the bills at each step of the parliamentary process. Both chambers only publish these as HTML and PDF documents, requiring the extraction of the desired information with automated . Transforming this into data requires parsing the various layouts and scraping the legislative structure and text out of each document. At this stage only half of the work is achieved; the biggest challenge is to autocomplete the many missing pieces within the text . For what are likely practical syntaxic reasons, the texts published by the two chambers often do not include unmodified articles or pieces of articles, letting the reader, hence our robots, crawl though a maze of previous versions to look for the actual missing pieces of text.
Once the data was finally assembled and the code published as a free software, it could be redistributed for anyone to reuse as an open data API tree giving access to each data file used to display the visualizations, as well as all source data that was assembled to generate it.
Following the GitLaw ideas, it only seemed natural to also host version-controlled git repositories for each one of the bills: as within a computer programme project, a bill’s articles become text files, and at the date of each step of the parliamentary process, an institution commits a new version for each article it modified. Using the free software GitLab, anyone can browse the repositories on a web platform, and, like on GitHub, propose their own amendments by “forking” a project and submitting their “pull request”.
What’s next?
Still, the automated handling of the sources’ discrepancies remains imperfect. Our robots fail today over 125 of the texts promulgated since 2010, which means our corpus represents only 70% of the considered texts. We remain confident that we will soon be able to process the majority of the missing bills on one hand, and further on to integrate texts during their on-going adoption process, allowing anyone to access the detailed version of a text with the proposed amendments during the debates. If some of the parsing errors are clearly identified as procedure issues for financial laws for instance, some exceptional cases will certainly reach the limits of automation, as for this erratum we encountered which further “amends” the adopted text that was published.
This whole work will always retain a variety of complex challenges unless the institutions step forward. This is only one example of the many reasons why Parliaments all over the world should progressively migrate their legislative processes towards fully integrated routines into their information systems. Let’s just imagine how both the institutions themselves and the societies they serve would benefit from the positive externalities which can only emerge from parliamentary openness and transparency.
Anyone curious to see more on the subject should feel free to browse through the hours of video captation of the latest Open Legislative Data Conference :)