Open Source Code: 100% Code Release - By Default

[rafael5]

The 20% OSS release of code makes no sense from any technical or administrative point of view. This creates both unnecessary administrative burden and confusion, an is of little technical value:

1. Unnecessary administrative burden and confusion: It will take more effort for agencies to determine which 20% of code to release, rather than just releasing ALL of it. Who it to determine which 20% gets released? How is this to be determined? Why 20%? Why not some other %? Is this 20% a cumulative statistic over time, so that agencies must publish 40% the second year, and 60% by the third years? This raises too many questions, none of them of which are useful expenditure of agency resources. The number should be simple, clear, and enforceable: 100% of all taxpayer funded code, by default. [with exceptions, of course].

2. Technically of little value: By definition, software will be broken and nonfunctional if it is not released as a complete functional unit. In other words, even if 99% of the code of any given software or module is released, it is still 'broken' and nonfunctional, rendering this of zero value to any other federal agency for re-use. Any source code needs to be released in FULLY FUNCTIONAL AND TESTABLE UNITS of code. That means, again, 100%.

THEREFORE, the wording of the OSS release of code should read as follows:

"All (i.e. 100%) of source code of federally-funded software must be released by default in an open source, publically accessible, version-controlled code repository, such that it remains available for download and use by all other federal agencies without restriction, with no redactions or omissions, in FULLY FUNCTIONAL FORM, unless there is an allowable exception for segments of the code.

If some sections of the code must be redacted for allowable reasons (cryptographic code, security code, IP regulations, or embedded data and metadata, such as personally identifiable information, URLS, network addresses, or other agency confidential data, etc.), only the specific segments of code and data may be removed. ALL REDACTED CODE AND DATA MUST BE IDENTIFIED AND DOCUMENTED. The documentation of what was redacted (with specific reference to the module and lines of code) must be included with the code, such that a third party could restore the functionality of the code with their own agency-specific data, metadata, or security code".

[Scotchester]

While I agree that the 20% requirement is less than ideal (I favor the open-by-default approach), I strongly disagree with the assertion that releasing code that is not a "FULLY FUNCTIONAL UNIT" has no value.

Releasing non-functioning or partially-functioning code could be valuable in that members of the public can now contribute possible solutions toward getting it functional. Also, I think you'll have a very hard time coming up with a useful definition of "functional". What does "functional" even mean in a software context? If it works as advertised but doesn't cover some edge case, is it "non-functional"?

You want to talk about putting a burden on agencies? Trying to determine what code is "fully functional" would seem to be a very slippery slope.

[rafael5]

I agree that defining "functional code" may be a challenge, as all software is a work in progress.

Neverthless, setting an arbitrary low bar - which is also unenforceable - is not ideal. There is no way to measure or enforce this. 20% of what? It is impossible to know if 20% was published because the 100% was never published.

If the intent of the 20% rule is a gradual publication of an agencies code, I fully agree. If this is the intent, a modification to this 20% rule which might be more reasonable and enforceable might be:

"Publish all code by default, but at minimum 20% of current agency code in the first year of the policy, with an additional 20% each year of this policy. Specific metrics of what constitutes % code (lines of code, functional modules, packages) shall be enumerated on code.gov"

Does that make more sense?

[Scotchester]

It does. That language would have sounded pretty good to me. Alas, the policy is finalized.

[rafael5]

Is there a version 2.0 of this policy coming out next year?

It would help the federal agency I work for immensely if there were some means to add supplemental technical language that would make this more enforceable. This is ongoing technical work going on by the code.gov group, no?

[mattbailey0]

Hi @rafael5 - while OMB policies can be updated as needed, they generally are not updated annually. However, per the final policy, the 20% requirements related to Open Source are part of a three year pilot. The pilot will be evaluated and extended, discontinued, or modified no later than two years from the date of the policy issuance. Here's the exact language relevant to that:

Unless extended or supplanted by OMB through the issuance of further policy, the pilot program under this sub-section will expire three years (36 months) after the publication date of this policy; however, the rest of the Federal Source Code Policy will remain in effect. No later than two years after the publication date of this policy, OMB shall evaluate pilot results and consider whether to allow the pilot program to expire or to issue a subsequent policy to continue, modify, or increase the minimum requirements of the pilot program.

With regards to the need for added "supplemental technical language" you mention, if I understand you rightly, The policy calls for that explictly. Here's one example of that language:

Within 120 days of the publication date of this policy, OMB shall develop metrics to assess the impact of the pilot program. Additional information on these topics will be available on Code.gov.

You can find both of these snippets in section 5.1 of the policy

[rafael5]

The supplemental technical language I was referring to is the following from Section 5.1: Pilot Program: Publication of Custom-Developed Code as OSS

"Agencies should calculate the percentage of source code released using a consistent measure—such as real or estimated lines of code, number of self-contained modules, or cost—that meets the intended objectives of this requirement. Additional information regarding how best to measure source code will be provided on Code.gov."

This is the centerpiece of the entire policy, yet it does not define its own metrics for success. Instead it defers these definitions to a yet-undefined metric created by another entity.

Do you have any when the team at code.gov is going to create this technical language?

[mattbailey0]

Yep! Per section 7.3 of the policy Code.gov is required by the policy to be launched "Within 90 days of the publication date of this policy."

[rafael5]

The 90 day gap where the code.gov team has to develop the technical implementation plan, metadata schema, and concrete reporting tools and mechanisms will leave only 30 days to implement the policy.

Perhaps better to re-word this to allow agencies 120 days to start implementing the policy after code.gov has completed all of its technical foundation work, and developed a robust, clear, and implementable schema.

Even 120 days is too little time to implement this policy if agencies are not already using modern code repositories (like Github) where they can automate the cataloging of their software resources with the Github API.

If source code is not in such a centralized repository (which it is not in most agencies) there needs to be additional policy that states that agencies need to first establish an agency-wide code repository.

[gbinal]

^^^ is also being discussed in GSA/code-gov-web#19

[johnmod3]

It would be great to see a few agencies take a real leadership role and release all of their code, no need to just meet the minimum

Sent from my iPhone

On Aug 26, 2016, at 14:44, Matt Bailey notifications@github.com wrote:

Hi @rafael5 - while OMB policies can be updated as needed, they generally are not updated annually. However, per the final policy, the 20% requirements related to Open Source are part of a three year pilot. The pilot will be evaluated and extended, discontinued, or modified no later than two years from the date of the policy issuance. Here's the exact language relevant to that:

Unless extended or supplanted by OMB through the issuance of further policy, the pilot program under this sub-section will expire three years (36 months) after the publication date of this policy; however, the rest of the Federal Source Code Policy will remain in effect. No later than two years after the publication date of this policy, OMB shall evaluate pilot results and consider whether to allow the pilot program to expire or to issue a subsequent policy to continue, modify, or increase the minimum requirements of the pilot program.

With regards to the need for added "supplemental technical language" you mention, if I understand you rightly, The policy calls for that explictly. Here's one example of that language:

Within 120 days of the publication date of this policy, OMB shall develop metrics to assess the impact of the pilot program. Additional information on these topics will be available on Code.gov.

You can find both of these snippets in section 5.1 of the policy

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

[johnmod3]

While I am 100% for Open Source -- in today's era where even the CIO of Box.comhttp://box.com says aside from their "secret sauce" for their core deliverable they reuse existing micro cloud services for everything else -- why is the USG coding so much? Even Google, after three years and who know how many millions of dollars finally opted to go with a third party cloud platform for their HR solution vs. homegrown.

We are not in competition with each other; software is not a unique monopoly we should hold nor should we think we can code it better than the private sector. As more and more reusable cloud micro services come online, why again are we coding so much vs. remixing and stitching together via APIs the best of breed to meet the different public service missions?

On Aug 29, 2016, at 10:12, John Scott III <jms3rd@gmail.commailto:jms3rd@gmail.com> wrote:

It would be great to see a few agencies take a real leadership role and release all of their code, no need to just meet the minimum

Sent from my iPhone

On Aug 26, 2016, at 14:44, Matt Bailey <notifications@github.commailto:notifications@github.com> wrote:

Hi @rafael5https://github.com/rafael5 - while OMB policies can be updated as needed, they generally are not updated annually. However, per the final policy, the 20% requirements related to Open Source are part of a three year pilot. The pilot will be evaluated and extended, discontinued, or modified no later than two years from the date of the policy issuance. Here's the exact language relevant to that:

Unless extended or supplanted by OMB through the issuance of further policy, the pilot program under this sub-section will expire three years (36 months) after the publication date of this policy; however, the rest of the Federal Source Code Policy will remain in effect. No later than two years after the publication date of this policy, OMB shall evaluate pilot results and consider whether to allow the pilot program to expire or to issue a subsequent policy to continue, modify, or increase the minimum requirements of the pilot program.

With regards to the need for added "supplemental technical language" you mention, if I understand you rightly, The policy calls for that explictly. Here's one example of that language:

Within 120 days of the publication date of this policy, OMB shall develop metrics to assess the impact of the pilot program. Additional information on these topics will be available on Code.govhttp://code.gov.

You can find both of these snippets in section 5.1 of the policyhttps://sourcecode.cio.gov/OSS/

You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://github.com//issues/259#issuecomment-242818488, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AAVH5j2O4_iB7s5t4WOp45jGtP-06R4wks5qjzP2gaJpZM4JuMk-.

[trevor-vaughan]

TL;DR: Regulations are fun and cause a lot of rework (all my opinion)

@johnmod3 In part, I'm guessing that this is because there would be no way to guarantee that the micro-services that are produces are adequate, in any way, to meet the Government regulations and requirements for data protection and/or may run afoul of the FAR clauses behind donations of works to the Government.

Now, if we ignore the slippery slope of micro-services and talk about actual projects that are run, from source, within a Government institution, then this would seem to be an excellent target for reuse and pushing patches back to the community. Then, each organization that uses a particular project could certify the the project was brought into the organization properly and in accordance with the FAR and all applicable data protection requirements and that it has been tested on the following "X" NIST certified operating systems (waived if the system does not process sensitive data).

But, as to why the Government writes so much code....they have a lot of rules to live by and projects done from scratch are, by definition, supposed to be following those rules. Other projects do not and may reach out to other services that cannot be used with impunity.

I mean, heck, half the FOSS projects I try to use (that aren't core Red Hat provided) end up bombing when FIPS is enforced which is, per NIST 800-53, a requirement for sensitive data on all Federal systems.