- Steven Rich
- Sarah Ryley
- Annie Waldman
Public records requests for data come with their own unique set of challenges — from agencies that insist their databases can only export documents, to those that will only send a fraction of the fields that are disclosable under the law. We’ll give you tips on writing rock-solid public records request for data, and how to respond to the common excuses used to deny these requests. Topics will include: Common sources that prove the agency has a database and can export data; statute, case law and language to make it more likely you’ll receive data (versus a PDF, or even worse, scans of redacted printouts); how to negotiate the confidentiality issues that come with medical and educational data; and managing the mass FOIA request project. We’ll provide a sample data request, a few good war stories, and links to helpful resources.
- Make sure it's not already online.
- Treat everything you write as a legal document.
- All communications should be clear, concise, professional and well-formatted — you're writing for both people with no data literacy as well as the data folks who will generate the request.
- Followup phone calls with emails summarizing the conversation so you have a record.
- Request should preempt common objections.
- Negotiation can strengthen your appeal. Asking for data dictionary and record layout helps make an informed effort to narrow your request.
- Check validity of exemptions or falsehoods.
- Insist on determination in writing.
Acknowledge existence of online data in a request.
Common objections:
We don't keep it as data Can only export 5 fields Overly burdensome Etc.
Cite evidence that data exist and can be exported. RFQ/P, contracts can detail "current data environment".
Documentation:
Record layout, ERD, Data Dictionary, User Manuals, Report Writing Guides
You can look at released records to see what exists.
Tips for overly burdensome.
Data format does not limit disclosability.
Make sure you get everything, frame followups as missing records, not questions.
Annie Waldman
Main reasons: exempt under other laws, privacy, HIPAA, FERPA, other delays
- Research
- be friendly
- find internal data wizard
- ask first
- request itemized cost estimate
- negotiations are crucial
- know privacy restrictions
- research AG decisions
Claimed that b6 exemption and HIPAA were essentially the same. Under b6 court has to do a balancing test. HIPAA doesn't.
Applies to Health care organizations, any org that bills/transmits health care data, hybrid entities (schools, prisons, etc.).
HIPAA lasts even after someone dies.
Personally identifiable info.
But you can still get some data. And people are entitled to their own records.
Does allow for De-identified data. Can ask for personally identifiable data to be de-identified or dummy ids.
Can get limited or restricted-use data set with, e.g. ZIP codes. Will have to sign data use agreement.
If limited data sets are not enough, institutional review board.
FERPA
Can still get lots of data; "directory data".
FERPA also allows for de-identified data and limited-use data.
Steven Rich
Same request to a bunch of same kinds of agencies, or same request to different types of agencies.
Fired/Rehired
Law enforcement data:
- Not a ton of standardization
- Kept in different formats
- Databases no one ever uses
- Certain exemptions apply
Approach:
- Check the internet
- Ask the agency if they'll turn it over
- Familiarize with state's laws
- File request
- Pester like crazy
Understanding what you can get:
- Every state has different exemptions and legal precedent
- Generally speaking there's discretion
- Ask for things that are exempt
Request all the things:
- Request the same thing from everyone
- Mass email (mail merge) or tailored request with specific legal language
- Automate if you can
- Know your deadlines
Fight for the things:
- Request is a first step
- Create alerts for deadlines
- Pick up the phone
- Use a spreadsheet (or tracker)
- Be nice, then get mean
- Push for correct format
If you don't get things:
- Some battles are worth fighting
- Others aren't
- Always be appealing
- Prioritize based on your time
- Name and shame
- Apply pressure
Now what:
- Standardize
- Pick up the phone
- Get the collection methodology
- Understand that things can change
- Don't force it
Stuff you should look out for:
- 65,536 records
- Changing definitions in the data
- Changing fields in the data
- Definitional issues in general
- PDF data
- Missing fields (is a missing field a caveat for one you are using?)
Aftermath:
- Pick your targets
- Don't always focus on outliers
- Put everything in context
- Check legal issues
- Talk to experts
- Make what data you can public
University hasn't responded (sexual assault complaints), what should I do?
Talk to expert in the state, see if there are deadlines. They may have reported it somewhere else. Can file "constructive denial" appeal. Contact state attorney general.
Data that's statutorily limited who it can share it with. Can you get it by partnering?
Probably can't partner due to data use agreement, maybe can get it de-identified.
Disagreeing with legal precedents, what do you do?
Google state public records law. Attorney general can help.
When should you appeal?
Appeal immediately. Can get denied because of deadlines.
When do you standardize?
Immediately. Don't wait for them to come back.
Agency that charges a lot of money and also gives it to competitors?
Shame them. Technically nothing illegal with giving it to competitors.
Steven Rich is the database editor for investigations at The Washington Post. He's worked on investigations probing the National Security Agency, tax lien sales, asset forfeiture, policing and college athletics. He has been a reporter on two teams awarded Pulitzer Prizes, in 2014 for Public Service and in 2016 for National Reporting. Steven is a graduate of Mizzou and Virginia Tech. He was elected to IRE’s Board of Directors in 2015. @dataeditor
Sarah Ryley is an investigative reporter at The Trace, a non-profit news outlet that covers gun issues. Previously, she was the data editor at the New York Daily News, where her work triggered numerous criminal justice reforms. Her series on the NYPD's abuse of eviction laws, done in partnership with ProPublica, was awarded the Pulitzer Prize for Public Service. She has also taught data journalism and investigative journalism at CUNY and The New School. @MissRyley
Annie Waldman (@AnnieWaldman) is a reporter at ProPublica, working on both data and education projects. She is based in Brooklyn.
Description and speakers from official schedule