From the Memorandum: Government should be transparent. Transparency promotes accountability and provides information for citizens about what their Government is doing. Information maintained by the Federal Government is a national asset. My Administration will take appropriate action, consistent with law and policy, to disclose information rapidly in forms that the public can readily find and use. Executive departments and agencies should harness new technologies to put information about their operations and decisions online and readily available to the public. Executive departments and agencies should also solicit public feedback to identify information of greatest use to the public.
Transparency demands making the operations and information of government open to the public. The goal of transparency is not openness for its own sake but making data available in order to increase trust, drive performance, promote accountability, and enable citizens to make use of that data to comment, derive value, and take action in their communities.The transparency thread will explore how the Directive can promote transparency.
Topics may include:
Before reading this suggestion, please know that I am not a SME in financial management or reporting. The AGA CPR publication, Process-Based Financial Reporting (#10, April 2007) offers a transparent means by which to report both performance and financial information... both of which will become increasingly important over the coming 2 years. Is it possible that there is an agency poised (existing financial systems and leadership) to pilot this innovative idea using a subset of funding or in a limited geographic area? As PARs and AFRs morph to more transparent and easily-understood Citizens Reports, it seems prudent to consider a fresh look at traditional financial reporting. I have attached the AGA document for convenience.
The most obvious enabler of transparency is the agency website. However, we - like most agencies - are using RSS or planning to use RSS to enable government information to be most portable and usable by private and nonprofit institutions. However, this concept must extend beyond RSS to move government information across the board to more portable formats, such as XML, to support complex analysis and remixing of information.
We also often forget the services aspect of Web 2.0. Some parts of the private sector are moving quickly to a Servise-Oriented approach for businesses (for business-to-consumer and business-to-business). We should be looking at how we can bring government services online - not just information - and make them reusable by the private sector. In the future the public should not only own government information but also government services to a certain extent. The challenges here are significant: cost, technical complexity, security, etc. Some agencies CIOs are moving in this direction, but there hasn't been much discussion about this among the Web community beyond discussions about gadgets and widgets.
Thanks for the great post, [Name]. Can you say a bit more about the financial, technical, and security challenges you referenced in your post (especially as they impact transparency efforts)?
The discussion about re-usable services extends from the discussion about reusable information. If transparency is in part about the citizen owning government information, a logical extension is that the citizen should own government services (and be able to build on top of them). At the core of the concept is Service Oriented Architecture, an approach to architecting the enterprise to implement decoupled business services (transactional, process, or data services) instead of huge monolithic applications. I believe a Web Oriented Architecture is a derivitive of SOA and is a foundational technology for many of the current commercially hosted Web 2.0 apps. There are significant long term benefits for architecting the enterprise in this way, especially in terms of improving the agility of the enterprise. However, there are significant setup costs, including the cost to model the business processes and data; the costs to implement the supporting infrastructure - primarily integration middleware; and the costs to construct the services. The technical challenges stem from the fact that it's a compleletely new way to think about IT design for the business and for IT, and it requires knowledge of Business Architecture, Information Architecture, and new types of technologies (very geeky stuff). The security issues relate to how to secure the services and enable the appropriate authentication and authorization. I should say, I don't believe SOA is a necessary precursor to creating reusable data, but it would make it more efficient and effective in the long run. I believe NIH is further along in this area than most, but I don't know that anyone is thinking about it in terms of an enabler of Web 2.0. The private sector certainly thinks a lot about it though. Dion Hinchcliffe has a great blog that describes these concepts in layman's terms: http://blogs.zdnet.com/Hinchcliffe/.
Utilizing the web for services (i.e., applications for loans, services or products) is a great idea and has been implemented in a number of areas, but it's been haphazard at best. I think the problem with moving in that direction is as much a mindset problem as it is cost driven. Take VA's Loan Guaranty Service as an example. They have been utilizing the web to interface with the lending community, veterans and other service providers for years. They make it possible for a lender to process an application for Eligibility, request an appraisal, obtain a loan guaranty, upload quality review documents and then service a loan (where necessary), all of it on line. The veteran doesn't have to wait unless there is a problem. Loans can close in under 30 days where they had taken 45 to 90. Some loans have even closed in 10 days using these processes.
What Loan Guaranty was looking at was how their clients used their services. Where were the touch points and how could they lower the touch points (with each touch point, something could go wrong and cost veterans and lenders alike) to speed up the process for lenders and veterans and do so in a way that was safe for the network, veteran and lender. Before Task orientation really took off, they moved toward this type of architecture (just don't look at their regular web pages, they are horrific). Where Loan Guaranty has succeed, other areas of VA's Benefits area have fallen behind, often requiring paperwork to be mailed or phone calls to be made or appointments to be held despite having medical records or other means available for processing ratings for disability. The mind set has been, where's the proof instead of how can we make it easier for our population to give us evidence. These are two very different kinds of philosophies.
I hope that we can change [in this agency] our mindset and look for opportunities to connect to our citizens and other clients, but it won't come over night, not with the current group of managers who came up through the ranks with the mindset we need to change now. Once we do change how we look at things, I belive the rest will follow.
Create data standards and inventory like DC government did.
Run contests gov't wide to incentivize agencies to share/open up their data.
-Instead of the usual OMB approach to punish, give out awards/recognition to the top X agencies that are most successful in getting data online by X data.
-The "Open Gov't" Awards - where you get to go to the White House. Published on whitehouse.gov. Meet the President, etc
Run contests like DC's Apps for Democracy to publicize the data and how to use it:
-Follow best practices on how companies like Apple/Facebook publicize their APIs and platforms and provide some guidance on how to use it.
Maybe decide a name for the person who should be in charge of the data issues at each agency. Each agency should be required to assign a "Open Data Architect" (or whatever name you decide) who reports directly to X (CIO, head of agency, etc).
And definitely create targets and deadlines. By X date, X percent of non-sensitive agency data should be online. You could use other measures as well.
Whoever creates the targets and deadlines should meet with the agency's FOIA and Information Collection chiefs during planning, to discover lessons learned / issues that could facilitate / impede progress. I.e., integrate implementation of new targets and deadlines in light of what's known about existing information management processes rather than doing it in a vacuum, as has been known to happen. Some FOIA shops have made considerable progress in backlog reduction via proactive, strategic posting of oft-requested docs to agency web site e-reading rooms. The IC shops receive year-round public comments on planned data collection. If these folks have developed insight, it should be leveraged. [Name]
[Name], I concur with [Name] in appreciating your fresh approach to these issues. The more I learn about MAX (a Wiki that works as a networking tool), and about the folks already using this venue, the more I believe that we will very soon have the application that provides Public / Federal Collaboration. I was just in an online meeting with the MAX Collaboration Workgroup ([Name]), and it came to me that solving the Collaboration issues, will also contribute greatly to the Transparency AND the Participation issues for OpenGov (Name I have a assigned to the 21 Jan Directive) Project.
Transparency - because opening our networking and collab doors to our non-Fed partners gives us instant creds with those folks that we now only meet in GovLoop, FB or in brick and mortar conferences. Policy making sessions with open and transparent discussions with all interested parties following the School Board type of Open Meetings. Naturally some decsion processes or sensitive areas are held in Executive Session, but the Transparency is there.
Participation - for the obvious reason, that the Collaboration with Feds, States, Non-Profits, Citizens, Consultants, Educators, etc. IS participatory government.
This is both a transparency issue and a participation issue. The heading on the home page quotes the president
In order to do that some basic legislation needs to change. The Administrative Procedures Act of 1946 was great for the pen and paper day. It does not work today. The E- Gov Act of 2002 went part of the way with electroinic comments but now there are Web 2.0 tools that allow greater discussion and openess. The basic legal rules need to change so the tools of today can be applied to today's business process. The new direction also needs to consider what will be tomorrow. We do not want to be stuck with another out of date process in 5 years.
This is a great point, [Name]. In thinking about topics that should be addressed in the Directive, it would be great if you could enumerate specific ways in which the current legal framework falls short. What would you change? In what ways doesn't the E-Gov Act go far enough? What advice would you give to someone writing an E-Gov 2.0 in order to make sure that evolves with technological changes?
Good point. The Administrative Procedure Act (APA) does need to change. The most important change needed: rewrite it in clear, easy-to-understand language. The current APA is very convoluted, and even contains basic errors in paragraph designation. Rewriting it in plain language would be a major contribution. In terms of the processes established in the APA, there will always have to be some form of official publication for regulations and other government issuances. A revised APA may refer to an electronic format for these, but I'm sure there will always be a hard copy. What we need is a clearly written act that allows for the role of technology in facilitating publication of official information.
Along those same lines, the rewrite of E.O. 12866 that the administration is now engaged in should help clarify and simplify the rulemaking process. You can offer suggestions for the rewrite until march 16, 2009, by writing to this address: oira_submission@omb.eop.gov.
And if we're looking at laws, can we also look at how we handle privacy in a web environment where feedback loops are not only available but encouraged? Take blogging, where users often leave their names, where names are considered personally identifiable. VA's direction on blogging requires this:
b. VA Blogs Opened to Public Commentary. Any blogs that are opened to comments frompublic individuals must meet the following additional requirements:(1) Prior written permission must be obtained from appropriate VA management officials toopen blogs to the submission of comments from the public.(2) Blogs that permit the submissions of commentary for posting from the public mustprovide a comment entry field and a disclaimer on the blog that specifically states what mayand may not be posted at the discretion of VA. Comments submitted must be directed to theblog owner for review by designated staff and public affairs officials prior to posting.(3) If the submission field for comments or any other required fields ask for an e-mailaddress, name, or other personal identifying information, appropriate permissions to gatherpersonal information from public individuals must be obtained from OMB, and all privacy andforms requirements must be met (see other sections in this chapter). Any searches ofpersonal information will require System of Records status. An example of a disclaimer is:Leave a ReplyThis blog is governed by the general rules of respectful civil discourse and is used only topost commentary on this topic. It may not be used to submit requests for customer service;requests for customer service must be made to https://iris.va.gov; Please do not submit anypersonal sensitive information in your submission since this mechanism is not secure. VA willreview the content of all commentary and will not post any content that is not relevant to thetopic of the blog and/or contains inflammatory or defamatory remarks, profanity, personal orsensitive data, language that is or may be construed to be a threat, or containing links toinformation on non-VA Web sites that is deemed by VA not to be useful or relevant. Anycommentary that indicates illegal activity or a violation of law will be turned over to VAauthorities for appropriate action.Names are often considered personal sensitive information. https://iris.va.gov; Please do not submit any personal sensitive information in your submission since this mechanism is not secure. VA will review the content of all commentary and will not post any content that is not relevant to thetopic of the blog and/or contains inflammatory or defamatory remarks, profanity, personal orsensitive data, language that is or may be construed to be a threat, or containing links to information on non-VA Web sites that is deemed by VA not to be useful or relevant. Any commentary that indicates illegal activity or a violation of law will be turned over to VA authorities for appropriate action.
Recommendation - Appoint a Senior Executive as Director of Agency Transparency and Citizen Engagement in each agency. The majority of agencies do not have a senior executive in charge of all external communications. True, all agencies have Directors of Public Affairs, but most of these senior appointees focus on external communications with the news media and very few have responsibility for all of the external agency program web sites and contact/call centers. Each agency is uniquely set up and has different organizational structure for citizen outreach and engagement efforts. Larger agencies with multiple bureaus and numerous programs have multiple web sites and contact centers that do not communicate with each other. Yes, there are a few exceptions. However, most often there are bureau web sites and contact center that are managed by different organizations and they engage the citizen using adverse styles, technologies, and different Frequently Ask Question (FAQ) databases. Agency information and services disseminated to the public is consequently, inconsistent and contradictory. A senior executive reporting to the agency Secretary, responsible for all transparency and citizen engagement, would provide the following benefits:
- Ensure Obama Administration objectives for Transparency and Open Government are implemented in a consistent manner across the agency.
- Ensure information and messages disseminating from all agency communication channels comes from "one voice" and is consistent and accurate.
- Share citizen service best practices with all programs and channels so that all can learn and improve.
- Increase cost savings by reducing redundant citizen engagement programs and the sharing of market research and new technologies
- Build a cohesive agency team among all those actively involved in citizen engagement.
- Increase external public awareness of web sites and 1 -800 numbers through collective promotion and outreach.
- Increase internal agency awareness of the importance and need to be responsive to citizens
This is an excellent idea. I just like to add that transparency and openess in government starts with communications that are in plain language. A communication is in plain language if the audience for the communication can quickly and easily: find what they need, understand what they find, and act appropriately on that understanding. Someone in agencies championing clear communication and web 2.0 technology will certainly foster a transparent government.
[Name], what is your thinking in terms of the scale and organization of this Office of Agency Transparency (or whatever it would be called)? Is it just an SESer and 2-3 assistants reviewing annual reporting from programs and serving primarily as advocates/facilitators of efficiency, or a full fledged review office that has to sign off on every piece of communication coming out of the agency (or somewhere in between)?
How are the lines of responsibility drawn, especially in larger Departments? At [Agency] we have [Number] agencies, so I imagine each would have its own Transparency Officer. Would they report to the Department level Director, their Agency Administrator, or a CIO (or some combo)? How is this all resourced (both from an HR perspective - is this a career or political job - and dollars)?
I guess my concerns mirror a more general one I have (and haven't resolved) about to what extent we should be centralizing more and more info and activities. Ideas like creating a single US government website template, pushing everyone into standardized data schemas, and increasingly detailed policy from OMB and NIST have huge payoffs in terms of creating a seemless citizen experience, reducing costs, and creating opportunities for cross program integration. On the other hand, giving everyone the freedom to do things like implement their own CMS, build their own visualizations of the same data, collaborate with the open source community, and build folksonomies allows for more a more agile and innovative "let a 1000 flowers bloom" culture and results.
I know I don't have the answer here, but I think this is an important area for the overall Open Government guidance to address and provide a framework for future policy directions (getting to the issue of how we don't want to end up in a few years doing the same exercise and wondering why we are being held back by rules that are so five years ago).
I agree this is a great idea. Judging from the nature of most of the posts on this site, they would have to guard against this senior official being someone who is too much of an IT person. While IT will have its role, we need to be more concerned with the issues and the concepts of transparency, not the technology. An official such as the one you describe would need to be an expert in issues where transparency is critical to the agency (in my agency, this would be things like endangered species listings, critical habitat determinations, permit issuances, etc.) who also has detailed knowledge of -
- The Freedom of Information Act
- The Information Quality Act
- The Privacy Act
- Plain language
I cannot emphasize enough the importance of plain language as a tool for increasing transparency and, perhaps even more importantly, the perception of transparency. If the public can easily understand what we're telling them, they are more likely to trust us and feel that we're being transparent.
Transparency and Open Government memo ...
RE: Government should be transparent.
"My Administration will take appropriate action . . . to disclose information rapidly in forms that the public can readily find and use."
Architecturally speaking, you can have great intentions to deliver a product or service to the customer, i.e. the public (individual and corporate taxpayers). However, it is not only how you deliver it (an IT solution), but what do you have to deliver (data, service) and what does it take for you to produce it (business process, enterprise services, etc.) and get it ready for delivery (staging and production servers). You have to have a "clear line of sight" in all the Enterprise Architecture layers between the the most primitive/atomic element and the strategic goal/performance measure. Scanning the various pages of this wiki I see largely top level policy oriented discussions in the vein of the strategic layer and the IT layer.
RE: Government should be participatory.
"Executive departments and agencies should offer Americans increased opportunities to participate in policymaking and to provide their Government with the benefits of their collective expertise and information."
We intend to have the citizens participate. We do not do enough to allow our own Government employees to meaningfully participate in policymaking and provide the [Government] policymaker with the employees collective expertise in the process of providing the citizens with [largely information] products and services. Yes, we have employees in Work Groups (quite often through several layers and mostly in the IT layer) that inform the policy makers. It is more a question of HOW we work than what we do. It is a matter of hierarchical versus "flat world", collaborative strategic partnership type of relationships and the antiquated ways of distributing and modifying information by e-mail attachments instead of web-based workspaces with collaborative tools available regardless of geographic, organizational or jurisdictional boundaries to the members of a collaborative process or project.
RE: Government should be collaborative.
"Executive departments and agencies should use innovative tools, methods, and systems to cooperate among themselves, across all levels of Government ..."
Once again, we have to start bottom up, inside out. We already know a lot about the top, the strategic level, especially the "outside facing" goals and desired performance. It is time to look to the inside and overhaul the old steam engine and allow the self starters operating it to use the rocket engine (the parts of which they might have gotten in the realm of "Shadow IT"), let them train the willing and teachable colleagues and make the rest take training designed and delivered by professional trainers who develop curricula and job aids based on the input of the self-starter SMEs. (Somebody already mentioned atracting and hiring young people. We already have a lot of young and young at heart. Unleash them!) The Government must disposess its leaders/managers of the idea that all they need to do is command those they lead in order to have a collaborative relationship with those whom they serve. The leaders must communicate with their followers, the managers with the subordinates, the old-timers with the newbees, etc. The pace of the technological progress (the Web is only about 5,000 days old) has put the diseases of bureaucracy under the magnifying glass: the natural tendency toward self preservation and status quo via looking for a good fit instead of searching for means to achieve actually needed and desired results through innovation, power sharing; inflexibility; no consequences for doing nothing, doing the wrong thing, etc. (I am sure you all have heard that "no good deed will go unpunished". It is horrifying, but it's true; people do say that.) Internal information exchange and production has to be sped up by the collaborative tools readily available today if we are to to have anything worthwile to disclose and do so rapidly on our outward-facing websites.
What I don't see here is any emphasis on individual accountability for government employees who hold up requests for information. I posted a bit about it here: https://max.omb.gov/community/display/OpenGov/Conversation+on+Transparency?focusedCommentId=273876131273876131 as it has been tried in other countries with a great deal of success.
The short version is that if I was personally charged $20/day for holding up a FOIA request, you better believe my whole Department would institute data management protocols with a quickness.
I too believe in accountability - at all levels of the Enterprise. The question is exactly WHO or WHAT and WHY is holding up the requested information. I am addressing a specific set of problems, i.e. the lack of deployement of existing or easily obtainable production efficiency enhancing tools: If there are tools the taxpayers paid for which would allow me to work more efficiently, but I am not alerted to their existence or am hampered by their not being deployed for and by all of my collaborators to use, then the fault and accountability for my not responding fast enough lies somewhere else. :(
I think you're spot-on, [Name]. But no one will ever know who was accountable, even for the decision whether or not to hamper you from using appropriate tools, unless there is accountability all through the chain. There have to be safeguards, and you should be able to show that you followed all the appropriate steps to access the information, and that should be enough. As it is now, we don't even have to show that, unless someone files a long and drawn-out lawsuit.
The key point here is that, at least in the model I described, such penalties follow a chain of responsibility, so that if you can prove that you did what you were supposed to - which is trivial now through email archives - the penalty goes down the line to the guilty party. And if you can prove you just have too much work to get to all your requests, well, that itself is useful information for the public to know.
Interesting. I've always had to be accountable. I am ready.
The only thing that is scary is that the adjudication system, unless independent of the Government - if any such exception exists at all - will of necessity be as corrupt as the system it tries to reform. There is no reforming a government. The only thing you can do is keep it slim and trim and put it on tight leash. The Founding Fathers knew it. ;)
Without a vibrant civil society and public oversight of the adjudication process, I think you're absolutely right. I guess this is where Participation (a standing Information Commission?) comes in.
The problem becomes lots simpler if you levy part of the responsibility on the transparency community. The goal should be to make the data as accessible as possible - meaning structured data that is easily consumable. If later the agency wants to create reports from that data, great! But the goal should be to get the data out there in ways that citizen interest groups can than take it, do mashups of the data, make iphone apps, etc. This is how the citizens will probably be consuming the data.
I work with trying to provide greater access to scientific data to maximize the government's return on its investment in scientific data collection within research and monitoring activities. The Federal Geographic Data Committee's content standard for metadata (originally introduced around 1993) provides a framework for documenting data sets, thereby overcoming one of the biggest concerns of the scientists - that their data will be misinterpreted and/or misused. And yet, 16 years later, doing metadata is still largely regarded as an unfunded mandate and many agencies have no (enforced) requirement to document data and no controlled archive for its long-term preservation. I believe part of the problem is that many research grade scientists are not incentivized to spend time on documentation, and it does in fact take time away from their research and publishing activities. I'm sure there are many things we can do to ensure the public has access to the raw scientific data produced by agencies. A few of them are: (1) update OPM's Research Grade Evaluation guidelines to reward scientists for documenting their data; (2) provide metadata clearinghouses and information infrastructures with searchable, web-based access to allow potential users of data to discover its existence and location (such as the National Biological Information Infrastructure www.nbii.gov and the National Spatial Data Infrastructure www.fgdc.gov/nsdi); and (3) encourage (and maybe even require?) the use of existing infrastructures to archive documented data sets to ensure that links in the metadata records are persistent.
A few the challenges we are facing that prevents public disclosure of our mission data include:
- Propietary data.
- Data that is sensitive, due to natioanal security reasons post-9-11. Individual data points may not be sensitive, but aggregated together it creates a potential terroist threat.
- Technology gaps
- No CMS to expose semi-structured data
- Structured data is within many DBMS throughout the enterprise. This problem is currently being worked on.
- Lack of financial resources to do the work necessary to make the data consumable, especally taking into account we would need to filter the data in a way to mitigate the first two points.
One continual message coming out of the Transparency Camp this weekend from the open source developer community is that they were complaining that they couldn't do free work to benefit the government. They do this all the time on open source projects, but want to find ways of doing this for government agencies. From there perspective, working for an open source project or working for a government agency both results in their work ending up in the public domain for the greater good. Perhaps if we can find a way around this issue, we could address your last bullet about financial resources.
One thought that came out was to see if the SmartBuy program could be levereged. That basically they would create some version of a non-profit creative commons organization that negotiates a really low ESI agreement labor rate for software coding - like under 10 bucks an hour or something for working on transparency initiatives. And then open source developers who want to help would in effect become consultants through the creative commons organization.
[Name]
Its fantastic to hear the open source community is so eager to engage. What do you see as the key barriers (legal or otherwise) that have prevented their participation thus far?
Hi [Name], they have gotten responses that they are not allowed to do free work for the govt. I will be talking to the [Agency and Position] next week to get better insight into this, and hopefully (assuming this part of MAX is still running past March 6) can report back with better insight. Also, just to update folks on this, I've done some investigation into this and have had more talks with some folks in the Open Source community. From a govt standpoint, there is a govt POC in our office looking into options with this - he has already talked with the Smartbuy folks, and in the process of contacting the office of procurement policy to figure out more options. It looks like there are two broad paths to pursue.
The first is to find a method to give open source developers a method to big on paid jobs via GSA - this seems like a relatively easy problem - they just need a holding bin type company to work through. I've already put the offer forward to some folks in Redhat to work this issue, but again, I think this is an easy thing to address.
The second is to in effect have a place for Open Source Transparency Projects that would allow resource-constrained agencies to post data transparency projects and challenges in a way that they could engage the open source development community to work for free to solve. [Name made] a more detailed post on this at Govloop at: http://www.govloop.com/profiles/blogs/open-source-government. I'm also meeting with some Transparency folks next Wednesday morning to explore this idea further. The hope is that we can quickly determine if there is really any interest for this idea, and to identify the barriers that might stop us from proceeding.
[Name]
While current federal acquisition regs prohibit Govt from accepting free products and services from vendors, I believe there is much we could do to potentially model commercial practices while at the same time open up the market to true competition and become more innovative buyers. For example, much work has been done on the ITI LoB - examples:
- have defined infrastructure areas
- have defined benchmarks for at least some of the infrastructure areas or "sub areas"
- agencies have submitted 5 year optimization plans
- there is a plan in development to move to service oriented infrastructure
- desire to identify criteria common govt wide enterpise infrastructure services
First step might be to take the "basic infrastructure services" as identified by the Gartner benchmarks and use an existing contract such as Networx to get something Gov-wide in place for the basic platform and pay as you go services - might require a competition and some upfront funding to get something established. There are models in place for this kind of funding mechanism today - think of the eGov projects and the telecom transition fund. Could also find a way to share as much of the infrastructure requirement information during the acquisition planning phase to get vendor input on developing the requirements. Perhaps use of a collaborative wiki site to allow comment and discussion.
The basic platform would need to be "open" so that other vendors could build additional services (think in terms of widgets) and offer those services pay as you go also in conjunction with the basic platform. Rather than putting up a competition for additional services, the metrics data and other technical data should be released to the public. For the infrastructure services shared across the enterprise that have been identified, vendors could decide (meaning the Gov would not put out a formal RFP) if they want to offer services that would be interoperable with the basic platform and build according to the benchmarks. Agencies could then again decide what additional services they need and buy pay as you go.
Hi [Name], the way we are looking for avenues around the govt acquisition regs is to list this as an open source project in which the process and software for extracting the data becomes open source and is usable on other govt transparency projects (as well as anything else in the public domain), but as a side effect, the data itself now becomes available for public access & consumption. Again, we certainly haven't touched all the bases in this effort yet (we're just starting the process in fact), and are still just trying to run down what is possible. We certainly have gotten some interest from the open source sector though, as well as some support from some of the technology luminaries.
The basic thought - that people are already doing lots of stuff for free to aid open source efforts, should also be able to help their government - seems to make sense. Its just a matter of understanding what if any options are there to make this approach possible.Regarding your infrastructure services approach, are you envisioning this basic platform as an open source effort?
The infrastructure services approach is just an idea I had, nothing official and no action being taken but yes, I was thinking in terms of a potential open source.
In looking at [the] comment above that "doing metadata is still largely regarded as an unfunded mandate and many agencies have no (enforced) requirement to document data", this is the type of thing I would hope that an open source partnership between government and industry could bridge. My guess is in the future there still may be problems getting funding for this sort of thing. Hopefully we can find ways around the procurement rules that can help with a govt-open source partnership.
Sorry, one other challenge. Most program data is contained in unstructured documents or not even written down. So, we have to first get it out of peoples' heads and second get it in a form that is both easy-to-use for the data provider and portable (system-to-system), so that the public can get at it. Or for data that just can't be put in a structured form, help agencies with the process for applying metadata/metagging principles, which is time-consuming and costly.
[Name] is completely correct about the unstructured/non digital nature of a lot of agency data. I think one hugely under appreciated issue (at least in my little over a year as a Fed) is usability - on the public side, but especially on the side of the folks who should be putting data into the systems. I worked on a project with [Agency] that managed to put just a tiny bit of resources into doing a usability review and implementing a few of the recommendations. The result was an improvement in both the quality of the data (our quality metric went from about 85 to over 98) and the overall use of the system (field offices are supposed to update their data monthly, but actual updating went from sub 50% to consistently over 80%).
And metadata - and automating that - will continue to be a huge issue. I went to a FIRM Council meeting in January where Air Force presented their work on building an automated system and it has been a huge undertaking that frankly I don't think anyone outside the military community could support financially.
[Name] - can you give us a few examples of the unstructured/nondigital data you're concerned about? What, in addition to automatic metatagging, would help to get at this problem?
Our agency, like many others, haven't yet implemented a robust Web content management system to support mettagging in an efficient way. So, we are still very dependent on legacy web authoring tools. Even when we implement Web CMS, the handling of the metadata is often not thought through and/or not enforced. I see real value in a combination of government content managers tagging content as "experts" and citizens creating folksonomies around that. But we would have to educate our contributors, create rules and systems that enforce metatagging, and implement content management systems that enable this. Or at least I would think that's the way it would work, since I haven't personally implemented a CMS that supports folksonomy creation.
Unstructured data can be anything from newsletters, fact sheets, budget documents, process documents (SOPs), policies and regulations, grant applications, and just about anything else you can think of. It seems that people in government are more inclined to publish documents instead of Web content, since the navigation of PDFs has improved. By the way, some people would say that there is no-such-thing as unstructured data, since any document also has structure. So, I'm using the term to mean data that isn't stored in a database.
There is probably other data about business processes in particular that isn't anywhere but someone's head. This becomes a huge knowledge management problem. My guess is that non-digital data is data on paper and not even in a system. I'm struggling to think of an example of that right now that would be made available to the public.
In our specific case, some of the non digital data was things like "how many miles of stream have been mapped" and "what day was the public meeting held" that could easily be digital (the project manager and contractor knew them) but were not primarily because of cultural, usability, or "requirement" (eg a formal status report is not a deliverable in the contract, so we aren't going to write one up) issues.
Another pile of non digital data is everything that goes on in meetings, over phone calls, and in other analog settings. Most of the things above are a subset of this, but I guess I would just note that as more of these become digital it is going to be important to connect the IT and records people together so that we make sure things end up in the correct streams.
For non structured data, it was things like emails, word documents, screen shots of graphs, and poor scans of something that started as structured digital data. My cautionary note here is just that you can plan out a great XML schema, but if you do not 1) train the people putting the info into it and make it usable, and 2) have a commitment and plan to monitoring data quality, it won't necessarily stay structured (especially in the non-scientific pieces of government).
One other thought I just had and wanted to throw out, is to what extent do our transparency efforts need to be compatible with legacy data? Is it ok to just say "search for everything created after 2007" or should we be planning to either create systems that accept the old data or retrofit it to the new structures?
[Name] - apologies for the quick reply (I had a nice lengthy one that Atlassian ate). Our best non digital data was the stuff in our project managers heads (how many stream miles mapped this month, what percent complete is the project, what date was the public meeting held) that was picked up through analog interactions - visiting the project site, attending a meeting, phone conversation with the contractor. Some of this did eventually end up digital, but often not until the mandated annual progress report or after completion (which from a transparency perspective is usually too late to do anyone any good). The reasons ranged from cultural (we don't do it that way) to usability (I couldn't figure out how to log in) to enforcement (the contract doesn't require me to produce any monthly data, so please leave me alone). Over time I see more of those analog interactions becoming digital, which will make them easier to capture. But, it will also be important to include the records managers and lawyers to make sure things are being put into the right streams.
Non structured was the usual suspects - word docs, scanned images, emails, etc. Auto tagging will help here, but will also need some help from things like OCR, people, and policies. I would also add that even the best hatched XML schema can go awry if not coupled with training and a good data quality plan to monitor and enforce standards (especially in non scientific areas).
One question I would throw out there is to what extent should our transparency efforts support legacy data? Is it enough to just have "search everything from 2007 onward" or do we need to build systems with backwards compatability (or even reprocess the data to fit it into the structures)? Obviously in a perfect world it would be everything, but given limited resources how should we be prioritizing?
One of the challenges of making data transparent to citizens is that the issue is not well understood. Bear with me while I explain.
First, let's revisit the purpose of transparency. The purpose of transparency is to increase public trust. We're making a leap of faith that more data means more trust. More data could lead to either no additional trust, or even decrease trust. The criticisms that lead to the OpenGovernmentData initiative in Sebastapol, CA. are well intentioned and represent a valuable contribution to transparency, but we need to understand how they may, or may not, lead to higher public trust.
Second, citizens deserve value for their tax dollars. OpenGovernmentData is part of that value, but there's no consensus even within the technology community about what form represents value. Data comes in all shapes and sizes: ASCII, Feeds, Tweets, Relational, HTML, RDFa, Natural language, etc. They're all different and represent diverse value to different constituents. Owen Ambur values data in XML; Tim Berners-Lee values data in RDF; John Sowa values data in Common Logic and Edward Tufte values data in revealing visual representations. I like what Clay Johnson, of Sunlight Labs, articulates as the priorities for OpenGovernmentData:
1. Provide data in its rawest form;
2. Provide machine readable APIs; and
3. Add value by creating a user experience.
But we need to understand the constituents served by each of Clay's priorities. #1 and #2 above satisfy one set of constituents. But, Tim, John, Owen and Edward want something different. I've been using data from the government in various technical and econometric projects since the 80s and it takes significant expertise to add value to raw data. Part of the promise of crowdsourcing is that either social production, or an emerging market will form to create value around #1 and #2. We don't understand yet how this market will form and some experts do not want to be part of the crowd. When it comes to data sharing we'll need some experts. And some econometrics around that marketplace, social production and how it may or may not deliver value to citizens for their tax dollars would be useful.
Third, Clay Shirky made a good observation recently that more data may serve the advantage of well organized interests and poorly organized interests may become disavantaged. Could small business be disadantaged through OpenGovernmentData? Its possible that big business would have an advantage with number crunching.
Fourth, there's a paradox in data availability. More data may not show that the conclusions government reaches are consistent with the data. If they show the conclusions are inconsistent and that leads to better government, great. But, what if no matter how much data is made available, there's no evidence that the government can draw valid conclusions? This would most likely decrease public trust.
When more data leads to higher public trust, it's great. But we shouldn't equate data with public trust. Don't get me wrong, I write mashups and have a background in semantics and such. For me the more data, the more fun, but we should also remember that data comes at a cost. Boxes need to be up, salaries paid, service levels met, etc.
Anyway, to further the debate around the important issue of Open Government Data, [Name has] submitted a revision to the FEA Data Reference Model that will cause the debate to be heard in the AIC Data Architecture Subcommittee (DAS).
I suggest that folks interested in this issue, get in touch with their DAS representatives and I welcome other folks thoughts here on Max.
[Name]
Is the purpose of transparency to increase trust or to increase accountability?
The distinction is not just semantics, but it also highlights a difference between focus on the needs of the "sender" (the USG in this case) and focusing on the needs of the "receiver" (the public). From a sender perspective the purpose of communicating is to persuade (e.g. to get someone to trust you), from the receiver perspective, the purpose of accessing information is to make a decision.
While we would hope that making the government more transparent would increase trust, as you note, it make lead to a decrease in trust if the data doesn't appear to match the conclusions. In that case, we would be held accountable to explain the discrepancy.
The focus, however, has to be on the public.
The model that should be looked at in making data available (internally and externally) is that of a decision-support system.
Relevance is indeed the question that applies to all data. A colleague who just joined my team a few months ago asked me this morning how we used GPRA data. His question was prompted by ongoing reports on a study we are conducting to improve our data quality assessment process. I had to admit that so far, the market for performance information appears to be oversupplied. It isn't really clear which decisions require these data. Five years ago, GPRA data in general may not have been useful. I'm not sure that case would hold today. Any problems with supply will not be solved until we understand the demand side.
However, I do not think we serve U.S. citizens by considering Congress the customer in this case. That's where transparency comes in. We measure results not because they form the basis for budgets and appropriations, but because they represent our best effort to account for the outcomes. Whether that information leads to change is up to voters and policy officials.
I think there are some (technologically) easy ways to work on issue #1 (raw data). Government agencies who move information up the chain (or silo or stovepipe) are constantly whittling and refining. This makes sense, for the heads of agencies cannot process all information produced by, nor knowledge maintained by, their entire organization. However, in many of these instances, we do not maintain the raw (first draft) of information.
This gets at a different kind of transparency, I think: not just transparency to the public, but transparency within a hierarchy, which of course can be a tricky subject to tackle. However, I think traditional processes can run parallel with new ones that aim toward crowdsourcing and knowledge management. Take the Citizens' Briefing Book project as an example: the President still received his traditional briefing book, alongside one that was created differently (in this case, crowdsourced).
Here's an idea that all agencies could make use of, I think: weekly reporting. Instead of emailing Word documents up the chain (and losing data all along the way), let each employee post their weekly achievements in an archived, shared forum where two processes could take place: 1.) managers could tag items that they wanted to send up the chain in the traditional manner, and 2.) other employees could tag or rate items which they think should be brought to the Secretary's (or Director's) attention. One method is traditional; the other employs crowdsourcing. In the end, there would be two knowledge products produced for the top leadership to review, YET all the "raw" data (eg all weekly achievements) are preserved and archived for future use (trend analysis, community creation, background information, etc).
One obstacle to such an idea is resistance to the removal of gatekeepers. Managers have to trust their employees to post accurate weekly reports. However, if managers cannot trust employees to do this, how can they trust them to do accurate work each week? If there is a personnel issue to deal with, we must contend with that properly instead of using editing processes to mask inadequacy in the ranks. If we're more transparent with one another in the organization, we're going to unveil much value at very little cost.
Government employees with 20 or more years of service are an obstacle to making program data transparent. Many want to:
- Hold on to the paper, instead of using electronic files,
- Travel to meetings, instead of webinars, etc, and
- Keep data hidden from employees with less tenure, instead of sharing their knowledge.
Whew! Thank Dog I only have 19 years! ;) I do agree with some of your concerns, but I don't believe that this is related directly to tme in service. I have met some very young dinosaurs in my time as a Fed. We have a real problem with a large percentage of the work force, being stuck in the box, whether that is the cubicle farm where they work, or the mindset of routine.
Saw a great quote yesterday "same people + same process + new technology = same mess". Definitely need to shape thinking, change culture and eliminate redundant, non-value adding processes.
Ditto, Ma'am! This technology is so good, and mature, that it almost seems too easy, and that is exactly part of the problem. I have spent the better part of my IT Support career trying to convince people that 'one button' solutions are not real. I've built wonderful information processing systems that died on the vine when the Data Owners took no part in the ongoing care and feeding.
We must reshape the fabric, the culture of the Federal Workplace, starting with building an empowered, inspired, mobile staff and manager corps. And it's not age or time in service, but an inherent staleness, a comfort of only doing enough to get by. We got to shake it up...
I think that one of the federal government's biggest challenges will be how to convince social media to become Section 508 compliant. I am a big fan of social media. Some forms are more accessible than others. For example, blogs, twitter, and wikis are relatively accessible as-is, or can be made accessible easily (I think that's true).
Other forms of web 2.0 - youtube, facebook, and second life - are an accessibility conundrum.
It isn't enough for agencies to post information on a social media site, and link it back to the accessible version on their site. Section 508 is tied to the FAR, and states that agencies (paraphrasing) may not buy or use non accessible electronic or information technology - - unless doing so presents an undue burden. It is silent on whether using non accessible technology as a supplemental outreach method is permissible.
Section 508 is also concerned with ensuring that a disabled person employed by the federal gov't can do their job with accessible E & IT. If they can not accesss the social media sites - where all the buzz is now - then they are at a disadvantage because they don't have a level playing field. Especially if they are a federal web employee - they will have issues accessing Facebook, Second Life, and a host of other web 2.0 tools - and maybe they won't be able to be considered for a federal web position that involves web 2.0 activities. It seems so unfair to leave the disabled federal workers behind in this way.
Maybe the web 2.0 owners (google, microsoft/facebook, etc.) and the federal gov't could work on a voluntary product accessibility template for social media? Something like what the Accessibility Forum did for E & IT way back when Section 508 first became law?
My two cents worth,
[Name]
[Agency offices] have been been advised that they can not put their organization charts online. These are simple staff organization charts, showing staff titles, who occupies those positions, and their contact information. This information was useful for our clients (government agencies, business partners) and the public as well as [Agency] employees.
Our [Agency] states that there are legal issues and other policies in place that prevent them from being made public.
We have tried to work around this issue by creating a Service Directory, which is a listing of frequently requested services, staff titles and names, email addresses and phone numbers. Its not elegant, but it works for now.
I don't have a dog in this fight, but wanted to present this as an example of what was requested above: Legal and policy hurdles to full and rapid disclosure of information . . .
I wanted to clarify that these organization charts are often asked for by the public and the clients. I also saw this supported by metrics from the ACSI.
Importance of the User Experience: It's wonderful for the government to publish information, data, services, etc. on its websites, but if the websites and applications are not well-designed and tested, we've wasted taxpayer money. Agencies need to systematically employ proven methods that lead to measurable efficiency and effectiveness (and satisfaction follows) such as User-Centered Design. To do this, every web team must have access to usability specialists, web analytics specialists, and accessibility specialists (The Model Web Team). In America, what we value, we pay for.
Fortunately, user-centered design, including customer and task research, iterative design, and usability testing, pays for itself two to ten-fold or more within the first year if not sooner. To be effective and meaningful, "Transparency" must include making website top tasks "readily available and easy to use by the intended customers."
I second this point!
I've seen this happen at three separate agencies.
We spend millions gathering the best minds in the field, collecting the best data possible, performing bulletproof analysis, and then we publish the resulting mission-critical information on websites that are haphazardly thrown together and too-often obscure the information they are meant to deliver.
We need to consider the entire information system, and the user experience is an essential aspect of that system.
I think the biggest disincentive is that it can make candor and honesty more difficult (the youTube phenomena). This has all sorts of implications for sensitive dialogues. It is one thing if a cabinet Secretary has to publish their calendar online, but imagine when all their special assistants (or for that matter every Federal employee) have to as well. Not to say that people shouldn't have to face the light of day eventually and be able to explain without any reservations why they met group A or person B, but it can cause real challenges.
This gets to the whole issue of timing. There is always going to be a lag between when data is created and when it is processed and shared. But as technology improves the overhead in that process gets smaller, which makes it even more important for there to be some guidance/framework for grappling with it. Are we looking to have real time transparency? Should the associated data wait until after a decision is publicly made?
Timing is also very linked with the issue of sensitivity - my sense is the reason things like job numbers are held so closely is to prevent someone from gaining an unfair advantage by getting them first. But when everyone's ability to receive and process those numbers is being equalized, then maybe offering them in real time actually reduces/eliminates the sensitivity.
Most communications we have both to the public and intra-agency are formal in nature. Formal communications require staffing and coordination. Forgetting the policy constraints, it is a big cultural leap to leave all that behind. Risk aversion and almost subconscious risk mitigation sort of governs a lot of these interactions. The concern is if you accidentally step in a mud puddle, your project can be halted for a good long time (we all have done this). Caution is often the better part of valor in this case. The question becomes one of how to build trust.
Incentive: Being open while using plain language leaves more time for 21st century training, because we answer less phone calls and respond to less emails.
Disincentive: The public knows how you truly operate behind government walls.
Incentives: (1) Better solutions will result - my theory is the more brains working a problem the better - you have access to lots of experience and expertise. (2) If you make a mistake in an open and transparent environment, it will be seen as an honest mistake. It is easier to trust in an open environment. Mistakes made behind closed doors can easily be misconstrued and lead to trust issues. Development of better relationships is also another incentive.
Disincentives - (1) fear of looking like you don't have all the answers. (2) projects or programs in trouble can more easily hide in a closed environment. (3) Open and transparent means change which could be a challenge culturally, could mean lots of changes to regs, laws, policy, guidance etc which is costly and involves more work than people might want to do.
I would like to second [Name]'s comment here with respect to a disincentive to transparency. Forget transparency between the Government and citizens, it's extremely difficult to promote transparency between government employees within the same agency. The vast majority of the time, after someone asks me about setting up a "collaboration area" online - right after they ask how long it will take to set it up - one of their first questions is how do we protect the information so people outside our project team or sphere of control cannot get access to the information. Sometimes there is good reason, such as with truly sensitive information. However, the vast majority of the time it's about control of the information (power), fear of embarrassment, or some other reason that isn't really a justification. The right answer is that internal collaboration resources should be open and available to all employees - at least in a Read-only mode - unless there is a specific regulatory reason for it not to be. Too often we implement enterprise tools to foster communication, collaboration, and knowledge sharing, but then at the middle management level we implement artificial, unnecessary barriers and security controls for no good reason. You don't know what you don't know. With artificial security controls in place, you don't know if the reason you can't find the right information is because it isn't available or if it's because someone has hidden it. Want to promote transparency between the government and citizens, promote transparency within the government. Of course, like a lot of what we're talking about here, this is as much a cultural issue as it is a technology issue.
Quite right, [Name]. Transparency within the government is critical to changing the culture. Various agencies are making baby-steps down this road, such as Diplopedia, Communities@State, and the like. The only example of massive transparency reform I know of is India, where the biggest users of the transparency tools are the government employees themselves. First, they accessed their own personnel records to get what they were due in terms of salary and such. Pretty soon they were using it for even more productive uses. We make it way too easy to hide information from our fellow government employees.
To some extent, increasing transparency will increase our awareness of the varying sensitivity of information. We've seen the well-documented phenomenon of overclassification. If one works primarily in a closed environment, there is a tendency to put all of that information into that environment (to classify it) without taking the time to distinguish levels of sensitivity (or lack of sensitivity). However, in an open environment one is more likely to be think of the ramifications of one's information becoming public, and which specific items are more sensitive than others.
Working in the consular realm myself, I've had to distinguish between sensitive personal information vs. policy information, vs. information received from classified sources and that which we've obtained from open sources.
Putting internal documents, such as the Foreign Affairs Manual, online generally has made it possible for me to make it clear to the public that there wasn't one set of information for them and one set of esoteric information for me. Working from the same page with the public is generally more constructive.
Thank you for pointing out the varying degrees of sensitivity! Our program deals with 100% non-sensitive data, yet we come up against the same security obstacles as other programs dealing with much more sensitive data. One size does not fit all.
I'm in violent agreement with you, [Name]. There are no penalties for over-classification, especially in our department. I may sound like a broken record, but a model for individual incentives for transparency might be appropriate, here. I don't know what form it should take, but as it is, transparency is treated like just another unfunded mandate.
What topics/issues related to transparency should be addressed in the Directive?
I think there are a number of variables that need to be discussed in terms of giving guidance:
- Transparency for intra-agency operations - what is necessary? A good guiding principle here is that the goal of intra-agency transparency should be to make the information (not documents) accessible for those who need it to conduct work
- Transparency for the public - what is necessary? My guess is Kundra's DC notion of that operational data is also public data probably applies, but clearly the exceptions need to be well thought out.
- Rules and guideliness that address the "internal coordination before release" notion that most agencies operate under. This is clearly a cultural problem that even inhibits participation on an internal site like MAX. An idea that we in [Agency] try to socialize is the notion that we should "Post before process", meaning the data should be posted in an accessible form prior to undergoing an additional process point. To do this with the public, there probably has to be public visibility to the work processes that are used to produce public data.
- Internal to the agency, the rule should be "Access Replaces Reporting." This is a key notion that, if followed, results in significant modification to most reporting processes. The largest processes usually have the most reporting layers (the acquisition process in [Agency] for instance). If we start providing access to the working files or possibly in a XBRL file (taxonomy xml format - the guy writing the book with Kundra - David Stephenson should have the vision on how to do this) and build the reports as "views" from the data separately.
- Need to share overrides need to know: Easier said than done, but clearly the Intel community has made great strides in this area.
- Far better guidelines for when information should be tagged "For Official Use Only" or whatever variant of that the agency should use. There really should be significant, cross the board training on this which helps the agency craft a posture on this, and aids the individual in making decisions on this.
Here's a summary of what we've seen so far. Please keep contributing - especially ideas for the directive.
Financial issues are delaying implementation of a Service Oriented Architecture. Also, we need to find a way to allow open source developers to work on gov't projects. Perhaps leveraging the SmartBuy program?
FIOA officers have many lessons learned that should be drawn upon.
Wiki and other Open networking applications will greatly expand gov'ts interaction with the public - immediate transparency and collaboration benefits.
APA and E-Gov Act need to be updated to account for Web 2.0 environment and beyond.
Appoint a Senior Executive as Director of Agency Transparency and Citizen Engagement in each agency. This would resolve the problem that Agency information and services disseminated to the public is consequently, inconsistent and contradictory.
The key to transparency is communication that is clear and easy to understand.
A challenge is to overcome issues related to privacy and sensitive gov't information. Also, unstructured/non-digital nature of gov't data is a problem. What to do with legacy data?
We should think through whether or not more data really means greater transparency. Who is better off (small vs. large organizations) as a result of greater access to gov't information? Can't necessarily equate more gov't data with public trust.
Agencies are being told that they cannot put organizational charts online. There are legal hurdles to putting more information out in the public domain.
It isn't enough to just put more information out there - the websites need to be tested for usability and accessibility so that agencies don't waste taxpayer money.
Candor and open dialogue within gov't can be compromised by greater transparency. Caution and risk aversion often limit how open agencies are willing to be.
All -- when originally reading this thread, I hadn't expected the technical bent it would take... I was equating transparency with integrity, rather than IT. But many of you have helped me see that "transparency" can be both: integrity (the message...character, leadership environment) AND technology as medium.
Several responses are emphasizing the message, or integrity aspect. Bravo! As [Name] stated,
"I cannot emphasize enough the importance of plain language as a tool for increasing transparency and, perhaps even more importantly, the perception of transparency. If the public can easily understand what we're telling them, they are more likely to trust us and feel that we're being transparent."
So with [Name] message as context, please consider how the legal and medical information fields have adapted their language to a more straightforward tone... this is what I believe the American Public expects from its government as well. How do you trust those you do not understand?? In my original post, I attached a suggested format for Process-Based reporting that would increase trust in our stakeholders and in Congress. You don't have to read a thing, just flip to one of several matrices that demonstrate the idea.
[Name], I would also emphasize individual accountability for responding to queries for information. It falls under these topics: Mechanisms, processes and incentives for achieving transparency via the Directive (e.g. How should agencies be directed to comply?)
Legal and policy hurdles to full and rapid disclosure of information and how to overcome them
Success stories of increased transparency and its impactThe mechanism I propose is here, adapted from the Government of India context. The original law itself, the Right to Information Act, 2005, is fairly straightforward, and the definition of information is extremely broad. The protections for employees acting in good faith are included, as well as penalties for employees acting in bad faith.
The legal hurdles include possible Privacy Act and civil service contract implications from the eventuality of exposing the names and positions of individual government and contract employees.
There are a plethora of success stories of the use of the Right to Information Act in India, from the creation of dedicated online communities to an abundance of news articles.
In addition to the summary above (by [Name]), I would like to emphasize the importance of re-designing agency web sites with transparency in mind.
My impression is that many agency web sites have evolved over the last decade to emphasize dissemination of information on agency programs and key subject areas, which is all to the good, and most agency web sites now organize their information to some extent or to a great extent by the type of information consumer or user (eg, general public, student, scientist/researcher, professional).
Transparency is probably strongest with regard to web-based programmatic information, and probably also research results for those agencies that have a large research portfolio. Many of the scientific & technical agencies have given serious attention to transparency of results of Federally-funded research. Significant progress has been made on this front. I think significant transparency has also been achieved with regard to grant and contract opportunities. The web would seem to me to have enabled a substantial increase in transparency in these illustrative areas (and probably others), compared to the pre-web era.
My impression is that a similar, intensified effort may well be needed to increase transparency of other types of agency activities and certainly including agency policy study and policy making processes. Some of these are of course subject to the Rules of Admin Procedure, FOIA, etc. But I could envision a future generation of web sites that is more transparent about agency policy inquiry and formulation activities, as it is already about agency programs and research results, and the like.
There are of course serious issues to be dealt with such as handling/protecting sensitive information, processing increased input in a meaningful way, dangers of information overload, and the like.
I would also echo the earlier comments about the importance of usability and user-centric testing and evaluation of all such major web site re-designs and improvements. This would need to be closely tied to the kinds of web architecture and IT support mentioned in earlier posts. The challenge again is to find ways to increase transparency of these policy processes (and other agency activities), facilitate participation by those who wish to engage, find new ways to process, digest, and synthesize the likely increased input and feedback, and produce well balanced and documented policy studies and, eventually, results that are themselves transparent. In this sense, transparency and participation are closed linked.
This is a tough challenge, but I think the web-based tools, methods, and technologies now available can help find solution sets that were simply not realistic or feasible a decade ago, and were virtually impossible in the pre-web environment.
PS, I note that the Participation Thread has a lot of discussion about technology-based and especially web-based approaches that are being or could be used within a more Transparent agency operating environment.
I'm putting this under the "Transparency" thread, tho' I think this could go under the "participation and collaboration" thread also. This comment is excerpted from a paper produced by the Federal Web Managers Council, "Barriers and Solutions to Implementing Web 2.0 in Government": http://www.usa.gov/webcontent/documents/SocialMediaFed%20Govt_BarriersPotentialSolutions.pdf
While it references Web 2.0, underlying cultural issues are fundamental to transparency and innovation:
Cultural issues and lack of a strategy for using these new tools
Issue: Many agencies view the use of social media as a technology issue, instead of a communications tool, and management decisions are often based solely on technology considerations. In many cases, the focus is more on what can't be done rather than what can be done. The default approach should be openness and transparency. For this reason, agencies need to be prepared that the decision to use social media will have cultural implications throughout government. Some agencies have leadership and legal support and have shown that the benefits of using social media outweigh the risks; but many have not. The result: social media is not consistently applied across government.
Proposed Solution: The new Administration should communicate a government-wide strategy for using social media tools to create a more effective and transparent government. The new Administration's Chief Technology Officer (CTO) should require each agency to, within three months, develop their own social media/Web 2.0 communications strategy that describes how it will use their agency website and the larger Web to accomplish its mission, reach new audiences, and engage the public. The strategy should include resources needed to accomplish these goals.
Good comments and discussion today. I have summarized the main issues broadly. Please continue to post your comments and invite your colleagues to join in. The discussion threads continue through tomorrow, March 10th.
-Ensure the public has access to raw scientific data produced by agencies. Make the data accessible and in a usable format.
-Since there are prohibitions on accepting free goods and services from vendors, solutions are needed to help the open source community engage more.
-Transparency increases public trust (it can also increase accountability). Increasing transparency can increase trust; while releasing poor data will do the opposite. The data provided has to instill confidence.
-We need to work around the obstacles created by people who do not seek solutions or who are too hide-bound. We have to be more creative and think (and operate) outside of the "box" more.
-It's important to consider the varying sensitivity of information when we look to increase transparency. When we share information with the public, they should be able to see that we are using the same information set as we provide to them.
-While Agency websites have evolved in recent years, they still need to be redesigned with transparency in mind. Certain types of information (web-programmatic; research results) are probably strongest in terms of transparency.
-Usability and user-centric testing and evaluation of web site redesigns and improvements are important.
President Obama's initiative on transparency will be welcomed by the public. Its application will probably start with the recovery package, but could lead to a new age of how the government conducts its business, as well as how it makes such information clearly available to its citizens.
I wanted to point out one organization that has a tradition of transparency: OSTI (DOE's Office of Scientific and Technology Information). OSTI advances science and sustains technological creativity by making research and development findings more easily available to a wide audience: researchers at the Department of Energy, the American public, industry, academia and internationally. Here are some of their most useful tools:
DOE research http://www.scienceaccelerator.gov/
DOE technical reports http://www.osti.gov/bridge/
Federal Government-wide research results http://www.science.gov/
Federal R&D Project summaries http://www.osti.gov/fedrnd/
World Wide research results http://www.worldwidescience.org OSTI should be tapped to play an expanded role as the transparency initiative takes shape. If the Administration is interested in "transparency" in a broader sense than financial measures and seeks to provide "a window for all Americans into the business of the government," OSTI might be the agency to lead this endeavor, and could provide information like updates of DOE's R&D program and include easy to use features like a rolling window of DOE technical reports during a given period of time (and provide intuitive search capability and links to the full text of the material). This kind of information would show the public that scientific progress is being made each day at our national labs and other research facilities and give assurance that tax dollars are well spent.
As a follow-on note concerning focusing on the people accessing the data and information when creating transparency mechanisms... to as certain extent, from the user's perspective, it doesn't matter whether the data is structured or unstructured when it is stored, what matters is what can be learned from it when it is displayed.
A decision support model would be able to take data that in multiple formats, aggregate it, enable analysis and make the resulting information and intell accessible to the end user via a portal that masks the complexity behind it.
The public use these types of systems everyday when using news aggregators, or when comparing prices, or finding hotel rooms online. Data from a wide range of sources in a variety of formats is aggregated, tabulated, compared and or analyzed in a way that addresses the decision making needs of the consumer.
As projects like the Kansas Event Data System (http://web.ku.edu/keds/utilities.dir/KEDS.TPM.0606.pdf) have demonstrated, even unstuctured textual data can serve as an intell database.
Transparency within government helps make the government accountable to the public, but the act that the public is performing in looking at the data is "deciding" upon the credibility and worthiness of government information and action. For the public to feel that it has the data necessary to make an informed decision, the front end of an online, transparent government system should be structured using the lessons learned from the deployment of computerized decision support systems elsewhere.
Hi [Name], just to clarify, are you suggesting that the government should be the entity that takes the data from the wide range of sources to make the front-end view (my apologies if I misunderstood your post)? What the transparency and e-democracy folks are asking for is the back-end structured data, so that anyone with an interest can take the information to answer the questions they are most interested. Transparency to them means they get the underlying structured data prior to it being crunched and presented. So in this approach, you might have multiple decision support models (perhaps one being the govt) who use various approaches to mask the complexity to crunch & present the numbers. If we go this route, the data presenter will have the burden of proving they accurately presented the results.
[Name], I think you have the gist, but I'm not suggesting that government take its data from a wide range of external sources. Instead, I'm talking about how to go about presenting the data from a wide range of governmental sources. As you note, a lot of the discussion has been on structuring the back end. I'm suggesting that the focus be on the front end presentation. Structuring the back end will require a signficant amount of technological coordination and resources, and may still not result in information that is useful to the public (or to other internal users). It will simply result in data that is structured in a common fashion. Focusing on the front end, using a decision support model (i.e. how can the data be presented in a way that facilitates decision making), not only requires less compromise on the data collection side, but also helps ensure that the data which is made accessible is meaningful.
Looking at it another way, there is
- data collection
- data storage
- data analysis
- presentation (of raw data and/or analysis)
Much of the discussion has focused on collecting and storing data in a common format. That, may make sense from the systems standpoint, but from the side of the public or even of the internal analyst or decision maker, the important part is the analysis and the presentation.
Decision support systems focus on analytical tools and presentation.
Going the route of a decision support system doesn't put an additional burden upon the data presenter. I would argue that it does the opposite. It facilitates a more effective analytical and decision making process, thereby making it easier to present credible analyses of the data. And since the analysis is based upon an interpretation of the raw data, access to the raw data (complete transparency) should support the analysis.
On a further note, focusing on front-end decision support, rather than back-end data coordination as the mechanism for facilitating transparency also promotes greater collaboration. A front-end data aggregating portal with embedded analytical tools provides the flexibility to make data from anywhere accessible, without necessarily forcing an agency to change its data collection process. Forcing all agencies to structure their data in the same way will lead to discussions about which format is the best, and who owns the data, etc., without necessarily making the data more understandable to those who access it. Whereas focusing on the front-end, while requiring that there be some agreement on metatags and how data is linked, enables agencies to maintain ownership of their data, and collect it in the manner that suits their purpose best. They may move to a different structure eventually based on seeing the benefits of structures used by other agencies, but in the interim, with the appropriate front-end portal, the data can still be aggregated, analyzed and presented in a way that meets the transparency goals.
[Name] -- When you have a chance, take a look at www.nbii.gov. This is the portal to the National Biological Information Infrastructure, which is coordinated and managed by the USGS, but presents biology and natural resource data and information from a 250-partner federation from all sectors (fed, state, local govts, universitites, ngo's, private sector). The idea is to give decision makers access to as much data and information as possible so they have the biggest possible picture before they set policy or make resource management decisions. This in no way influences how USGS collects its data. The multiple data structures do, in fact, get in the way of true integration, but [they are] working on those issues. For example, some basic data formatting standards (not data collection standards) helps, and documenting the data via a structured metadata standard is really important. NBII, in turn, flows into a hemispheric-level system (InterAmerican Biodiversity Information Network) and a global system (Global Biodiversity Information Facility).
[Name], thanks for pointing out the NBII site. I had a quick look and the approach it lays out on the "About NBII" page is what I'm advocating, i.e. it "links diverse, high-quality biological databases, information products, and analytical tools maintained by NBII partners and other contributors in government agencies, academic institutions, non-government organizations, and private industry"
In this way you're able to have transparency, collaboration and participation without having to first get everyone to restructure their databases.
What has been the feedback from those who use it?
[NBII] has many very enthusiastic partners and supporters. But [they] also have [their] share of detractors. I hope I don't sound defensive here, but I've noticed that many detractors comment on NBII from heresay rather than from actually using it and being dissatisfied. I guess the concept of this very open collaboration is threatening to some. And I also know there are some things [NBII] could do better. But [NBII] have had budget cuts the last three years, so it gets harder and harder to get things done. This year we will implement a new search engine (contract just awarded to [Company]) and will update the look of the home page. Mostly, the users are very satisfied and, of course, always drive [NBII] to do more and more -- which is exactly what [NBII] want[s]!
Will the general public be able to access the MAX system? If not , which website will the public be able to post questions/comments, etc.? Also, whatever information is provided to the public needs to be easiy understood with some background information .
Query-based Transparency with Enforceable Timelines
I'm sure folks have been looking at models from overseas, but I'd like to share what experience I have from working with the Government of India's Right to Information Act, 2005. Below are a few salient points. I've stripped out the exceptions and caveats for clarity.
- Every government office (local, state, federal) has a Public Information Officer (PIO).
- Any citizen can request any information from any PIO, without stating a reason or providing anything but a return address.
- The PIO delivers the information within 30 days, or states that the information does not exist.
- If the PIO delays in responding, the PIO's own salary is penalized at a rate of Rs. 250 per day of delay.
- If the PIO can prove that she has been prevented from doing her job by another government officer or officers, the penalty goes to the guilty officer(s).
- If the PIO lies or destroys an application, the PIO is penalized Rs. 25,000 plus administrative penalties.
- An independent Information Commission adjudicates these cases, and has already levied penalties against many Indian public servants.
Now, like most transparency legislation, there are a variety of exceptions and caveats to prevent abuse of the system. While the system is far from fully implemented, India has seen some real results in combatting corruption. Even I saw results while working there as a volunteer paralegal, both in litigatign cases and training government officers. Such a system would be a great incentive for hold-out US officials to at least put their information online in easily-accessible formats to respond rapidly to such requests. I'd be happy to provide more details, or to brainstorm on how such a query-based system could be implemented to reform the current FOIA/e-Gov/Privacy Act system.
Change Rules about Use of Persistent Cookies on Federal Government Web Sites
I suggest revising the current policies that prohibit the use persistent cookies on Federal government Web sites -- without approval of an Agency secretary or immediate designee.
It IS possible to achieve the two equally important goals of providing high quality online services for visitors and diligently protecting their privacy.
Unfortunately, there is a lot of misunderstanding about benefits of persistent cookies and the privacy risks associated with cookies on government Web sites.
A cookie is a text-only string of information that a Web site transfers to the cookie file of a browser on a site visitor's computer's hard disk so that a Web site can "recognize" the visitor's computer upon return to the site (The Web site recognizes the visitor's computer by its IP address). Session cookies only last for the time visitors are on the Web site. (Federal agencies can use session cookies now -- without waivers.) Persistent cookies remain for repeat visits (until their set expiration dates) or until they are deleted by visitors. (All cookies can be deleted at any time by visitors.) Since cookies are non-executable text files, they cannot look into a computer, find out information about the visitor or read anything kept on the computer's hard drive.
Persistent cookies offer important benefits to both Federal government Web site visitors and Web site managers.
For site visitors, persistent cookies:
For Web managers and Web analysts (equipped with Web analytics tools), persistent cookies enable them to use data to gain insights into the needs of and differences between first-time and returning visitors. They also can help managers and analysts understand the characteristics of other visitor segments -- based on their actions on Web sites. All government Web sites are designed to support the mission and goals of the sponsoring agencies. By analyzing the actions of visitors on Web sites over time, managers can determine whether Web sites are achieving the agency and visitor goals. They can also recognize trends and changes in visitors' need for and use of Web content and features. Using these valuable insights, Web managers can add and remove content and features based on research and data, rather than guesses or hunches.
The rules about persistent cookies were written 8 - 10 years ago and the technology has changed dramatically since then. The current rules hinder, rather than help government agencies creatively and effectively meet the needs of visitors. Since a agency Secretary or immediate designee must approve a waiver to use cookies, the current waiver process is time consuming at best or impossible at worst. I believe it is time to allow all agencies to use persistent cookies -- that comply with existing privacy guidelines -- without waivers. Otherwise, I suggest allowing waivers to be approved by lower than Secretary-level agency officials who are involved in actual Web site decision making processes.
This, of course, is an obstacle for our program as well. Allowing persistent cookies would greatly enable us to track the preferences of our users and to accurately track repeat visitors, and to build more knowledge of our core customer base. The more we can interact with and build relationships with customers, the better we can serve them.
Each major Agency website should establish an area on its website dedicated to operational metics. These should include their budget, major programs, status of these programs, key metrics and other operational reporting. Much like publically traded companies must report on thier operations and health, Federal Agencies should indivdually report on their performance. This happens now to a certain extent, but only for limited, official metrics that must be reported to OMB. This can be greatly expanded upon to everyone's benefit.
I'd like to see this taken a step further, [Name], with operational metrics for offices, and even individual employees, such as our performance reviews. Already, Legistorm posts the salaries of our Congressional colleagues. Similarly, our bosses (the taxpayers) should know how productive their employees are.
The most important part of transparency and participation is the content. The tools will work put what is provide is most important. Who is the audience? General public, interest groups, industry. What do we want to tell the targeted audience?. What message that is clear concise and information. How do we organize the material and educate them on the content of the material. When do we post it and more importantly when do we take things down. Where on the web but where on the web one portal or create clouds of information from all agencies and departments.
I couldn't agree more, [Name]. Those are some of the reasons I'm putting forward a model for query-based transparency. Our first priority in transparency is answering the people's questions. We should have individual financial incentives for doing that effectively.
Currently we need to obtain OMB Clearance for surveys (using standardized questions) given to 10 or more people, as part of the Paperwork Reduction Act. Using standardized questions provides us with invaluable metrics on Web site use, profiles of the visitor populations, common tasks, success rates, and visitor preferences. The process for obtaining clearance is very time consuming and for many, intimidating. While in the past, OMB has granted generic or expedited clearance to various agencies, it has been temporary and very limited in scope. I respectfully request that clearance for surveys be distributed down to the agency level. This will enable much more rapid approval by individuals who are more familiar with the issues, while insuring that individuals with knowledge of statistics and survey methodology review the proposed survey.
Agree that accountability is in interest of govt and citizenry. The question then becomes accountability at what level? How can accountability be communicated with purpose and understanding? To what end? For example, if a common Govt service catalogue existed it could provide a means of collaborating among and within agencies and with citizenry and businesss. One great start is with current lines of business, egov initiatives and the VUEIT program on OMB website that makes transparent the over $71 billion in IT spending. If a crosswalk to a common service catalogue existed then the over 6000 IT mission and infrastructure systems could be clumped for definition of common purpose and understanding - say get 6000 system investment areas down to a portfolio of say 50 similar to the past exercise where over 500 previous and separate IT infrastructure business cases were collapsed into 25, making them more purposeful and understandable, thus subject to more strategic leveraging from both the engineering and social perspectives. Agencies could still retain budget and other control but the clumping under a common service catalogue would allow for input of engineering and social purposes ( get more secure and more accessible systems, assets and websites by applying reviews and actionable insights at the right level- provide ability to integrate web 2.0 and new media options, etc). As a common service cataogue is populated then both technical and social purpose can be addressed effectively and strategically. For example I'd like to see how many of the 6000 systems are actually collaborating on purchase of compliant hardware and software and services (see www.gsa.gov/feditam ). I'd also like to see some transparency from the social perspective so both govt and citizenry could see how many and what percentage of IT staff hired under service contracts are persons with disabilities (how many civilian? how many disabled vets? did you know IT contracts over X$ requires consideration of employment by persons w/disabilities?). How many and what percentage of systems, assets, and websites are accessible to persons with disabilities (see www.section508.gov ). I believe we need a line of business in a common service catalogue, just for federal websites so it can coordinate and collapse the gov web into something that becomes strategic and (yes Candi) governed. As a former bridge builder and engineer the analogy would go like this: I have 800,000 bridges to repair and maintain and I need to build 400,000 new bridges. Why not analyze those 1.2 million bridges to see what are the common types and categories - say 100 different types - then apply high end engineering and social purpose to those 100 types placing them into a common service catalogue for easse of understanding - resulting in a strategic game plan that is flexible and overall lower cost while meeting needs for safety and social purpose.
Just saw this news release from CMS Watch, based on their research...
http://www.cmswatch.com/About/Press/2009-CMS-Social-Gap/ Obama Call for Citizen E-Participation Will Encounter Technical Hurdles
CMS Watch Finds Gulf Between Web CMS and Social Media Technologies
SILVER SPRING, MD, USA-- US federal managers trying to heed President Barack Obama's call for more transparent and participatory websites face separate marketplaces for the supporting technologies -- Web Content Management ("CMS") and Social Software -- complicating efforts to align efficient web publishing and citizen interaction, according to research released today by independent analyst and evaluation firm, CMS Watch.
In a January memorandum (http://tinyurl.com/openg), President Obama called for both greater transparency in making information available online as well as more citizen participation in federal websites. CMS Watch research suggests that different technology platforms will be required to meet those two objectives.
"Washington is abuzz about new possibilities for federal web publishing, but actually executing on a strategy of greater transparency combined with public participation will be difficult," notes CMS Watch founder Tony Byrne. "Web CMS vendors and Social Software suppliers can talk a good game about offering a unified solution to this challenge, but their architectures have not caught up with their marketing here." Byrne adds, "In particular, agencies will find that their existing Web CMS infrastructures are not up to the challenge of public participation."
"Web CMS vendors have begun to incorporate community features, but in reality most CMS platforms are not built to ingest user-generated content -- they have more 'inside-out' architectures," argues CMS Watch analyst Jarrod Gingras. CMS Watch found that user-participation functionality in Web CMS tools tends to get activated primarily behind the firewall. "In higher-profile public settings," Gingras points out, "enterprises incorporating user-generated content have to pay more serious attention to security, scalability, usability, and access control considerations - where best-of-breed Community suppliers have a much better track record."
Meanwhile, although Community and Social Software vendors may effectively support user-generated content, CMS Watch found that these suppliers generally cannot provide the sorts of workflow and lifecycle services that a public agency needs for publishing official content beyond the firewall.
"There may be lessons to draw here from private-sector experience," notes Gingras, "where high-profile firms have tended to deploy separate platforms for web publishing and customer interaction."
"Everyone wants a single environment where tightly-managed and lightly-managed / unmanaged content can co-exist effectively, but the industry isn't there yet," concludes Byrne. "Any agency seeking to improve transparency as well as participation needs to keep a critical eye on the limitations of present technologies in a public setting, and consider a multi-vendor strategy going forward."
These conclusions stem from the most recent releases of The Web CMS Report 2009, in which CMS Watch interviews web content management customers around the globe to evaluate forty solutions in the marketplace, and the Enterprise Social Software Report 2009, which evaluates twenty-four social computing platforms. Both reports can be purchased online from CMS Watch (http://www.cmswatch.com). Free report excerpts available: http://www.cmswatch.com/Try/.