changes-journal.txt

1.  change all ca:isTargetof  to ca:isTargetOf
2.  change all bioj:(\d) to bioj:a$1

bioj:2007-48-UNDERSEA_HYPERBAR_M
bioj:2008-02-WATERBIRDS

bioj:a2007-48-UNDERSEA_HYPERBAR_M
bioj:a2008-02-WATERBIRDS

sed 's/bioj:\([[:digit:]]\)/bioj:a\1/g'

3.  Standardize  ontology location to http://james.howison.name/ontologies/.  This means changing http://howison.ischool.utexas.edu/

4.  Fix dc:isPartOf so that the ISSN is urn:ISSN:number not dc:number

dc:urn dc:1896-1851 ;

dc:urn <urn:issn:1896-1851> ;

sed 's/dc:\([[:digit:]][[:digit:]][[:digit:]][[:digit:]]-[[:digit:]][[:digit:]][[:digit:]][[:digit:]]\)/<urn:issn:\1>/g'

also change dc:urn to dc:identifier

sed 's/dc:urn/dc:identifier/g'

cleanup one or two (wish I'd included the X!):

sed 's/dc:ISSN1744-4292/<urn:issn:1744-4292>/g'

sed 's/dc:1365-313X/<urn:issn:1365-313X>/g'

sed 's/dc:1076-836X/<urn:issn:1076-836X>/g'

sed 's/dc:1303-829X/<urn:issn:1303-829X>/g'

sed 's/dc:1618-095X/<urn:issn:1618-095X>/g'
sed 's/dc:1365-294X/<urn:issn:1365-294X>/g'
dc:


---------------
Wed Feb 12 17:32:24 CST 2014
---------------


Then I deleted everything and re-imported:

CatherineCoding-singly_coded_AS_OF_01-22-13.ttl (md5: ccd8ccc2ec799d8466d7855d4141a9d0)
Julia-singlyCoded.ttl (md5: 23c789d8625f791dd68b61531c91ddeb)
journalCodes.ttl (md5: 105a323084a050c0266e75720eb65d0f)

Hmmmm, that gives 398 selections, not the 404 that I found before...ug.

60 articles with selections:

SELECT (COUNT(DISTINCT ?article) as ?articles_with_selection)
WHERE {
	?article rdf:type bioj:article ;
	         bioj:has_selection ?sel .
}

---------------
Wed Feb 12 17:39:40 CST 2014
---------------


SELECT ?strata (COUNT(DISTINCT ?article) as ?articles_with_selection)
WHERE {
      ?article rdf:type bioj:article ;
                  dc:isPartOf ?journal ;
	         bioj:has_selection ?sel .
      ?journal bioj:strata ?strata .
}
GROUP BY ?strata
ORDER BY ?strata

hmmmm, nothing under bioj:strata.  Damn, forgot to update the journalCodes.ttl stuff with the http://james.howison.name/ontologies/.

Delete everything and report.

CatherineCoding-singly_coded_AS_OF_01-22-13.ttl (md5: ccd8ccc2ec799d8466d7855d4141a9d0)
Julia-singlyCoded.ttl (md5: 23c789d8625f791dd68b61531c91ddeb)
journalCodes.ttl (md5: e10d0c07c0bd3433ce846817234bee70)

---------------
Wed Feb 12 17:54:18 CST 2014
---------------

Hmmm, seem to be 3 journals without strata?

These appear to be differently spelled in the isPartOf and the rdf:tupe bioj:journal statements.

http://james.howison.name/ontologies/bio-journal-sample#J_INTEGR-PLANT-BIOL
http://james.howison.name/ontologies/bio-journal-sample#J_INTEGR_PLANT_BIOL

and 

NAT-GENET (one dc:isPartOf) vs NAT_GENET (dc:isPartOf and rdf:type bioj:journal)

http://james.howison.name/ontologies/bio-journal-sample#J_THERM_BIOL

was just never coded for strata?

Hmmmm, there really isn't a consistent coding of - vs _ in journal name URIs.

Ok: actual changes:

sed 's/NAT-GENET/NAT_GENET/g'

sed 's/INTEGR-PLANT-BIOL/INTEGR_PLANT_BIOL/g'  # Note this changes a bunch of selections too.

Post these changes import:
MD5 (CatherineCoding-singly_coded_AS_OF_01-22-13.ttl) = a55483c3a6c246d7f0c936ff0cda041e
MD5 (journalCodes.ttl) = e10d0c07c0bd3433ce846817234bee70
MD5 (JuliaCoding-singlyCoded.ttl) = 67dac9287dc23b2d0b3230536e88e0c5

---------------
Thu Feb 13 09:36:40 CST 2014
---------------

Hmmm, J_THERM_BIOL.  I think this was a replacement for an article, perhaps a review article?

---------------
Wed Feb 26 09:57:57 CST 2014
---------------

Ok, selections.

	?sel rdf:type bioj:selection  shows 398
	
	?article citec:has_selection ?sel shows 285.
	
Why?

UNION of those two has 401 DISTINCT URLs.

Want ?sel where not bioj:selection.  Do this with a FILTER.  Found 3 URL typo issues. see commits: 508b603219d5cf10781fced4f8fdfd2c45e0049b and 47c207f0e0bd4426970a0ebd4a3de5ca34c5e6ac

---------------
Thu Feb 27 09:21:25 CST 2014
---------------

Articles have selections, some of those selections have_references.  The Reference receives codes, so it is a selection as well.

SELECT COUNT(DISTINCT ?sel) 
WHERE {
  ?article bioj:has_selection ?sel .
  ?sel bioj:has_reference ?ref .
  ?ref rdf:type bioj:selection . 
}
==> 126.
There are 126 has_reference statements?

The solution here might be to directly associate the has_reference with the articles.  Should make them easier to consider?  Going to depend a little on the specific questions, though.

SELECT (COUNT(DISTINCT ?sel) as ?selectionCount)
WHERE {
    ?sel rdf:type bioj:selection 
  	FILTER NOT EXISTS {
		   ?article bioj:has_selection ?sel
		}
}
==> 114
There are 114 bioj:selections not directly associated with an article?

So there are 126 - 114 has_reference statements that are directly associated?

Nope, all Domains of bioj:has_reference are bioj:selections.

hmmm.

SELECT COUNT(DISTINCT ?ref)
WHERE {
  [] bioj:has_reference ?ref .
}
==> 112.  references.
bah.

SELECT COUNT(DISTINCT ?direct_sel) 
WHERE {
  ?article bioj:has_selection ?direct_sel ;
           rdf:type bioj:article .
  ?direct_sel bioj:has_reference ?ref .
  ?ref rdf:type bioj:selection . 
}
126 selections with references.

Ok, that's easy, some references occur in more than one selection.

Q: how many direct selections have references? A: 126.
Q: how many references occur in more than one selection?

SELECT COUNT(DISTINCT ?ref)
WHERE {
  ?sel1 bioj:has_reference ?ref .
  ?sel2 bioj:has_reference ?ref .
  FILTER (?sel1 != ?sel2 )
}

Yup.  That gives 22. Which is the difference between 126 and 114.

---------------
Thu Feb 27 09:50:11 CST 2014
---------------

Ok, confirming vocabulary extent and possibility of typos.

Get all things used as a property.

SELECT DISTINCT ?p
WHERE {
 [] ?p []
}
ORDER BY ?p

Then visually inspect for near typos.

e.g.,

ca:isTargetof instead of ca:isTargetOf

bioj:full_quote / bioj:fullquote

Also notice lack of consistentcy between underscore_separated and studlyCaps.

Made the two changes above.

---------------
Thu Feb 27 10:05:04 CST 2014
---------------

SELECT DISTINCT (strafter(str(?p), "#") as ?localName)
WHERE {
 [] ?p []
}
ORDER BY ?localName

SELECT DISTINCT ?s
WHERE {
 ?s ?p ?o
}
ORDER BY ?s

but I only want those that match a namespace?
Can cut this down with regex.

found

citec:domain-publication
citec:domain_publication

replaced 2.

found

software_pubication

replaced 1.

---------------
Thu Feb 27 10:28:37 CST 2014
---------------

bioj:J_MOL_CATAL_B-ENZYM rdf:type bioj:journal ;
	dc:identifier <urn:issn:1873-3158> ;
	dc:title "Journal of Molecular Catalysis B: Enzymatic" ;
	bioj:strata "111-1455" ;
	bioj:hasAuthorInstructions "http://www.elsevier.com/journals/journal-of-molecular-catalysis-b-enzymatic/1381-1177/guide-for-authors" ;
	bioj:hasSoftwarePolicy "text" ;
	bioj:hasSoftwarePolicy "url" ;
.

Doesn't look quite right.  Are the bottom two just left-over templates and you meant that that policy doesn't have any Software Policy?

Found

urlhttp://www.biomedcentral.com/bmcplantbiol/authors/instructions

removed the url part of it.

---------------
Thu Feb 27 10:31:02 CST 2014
---------------

Email back from Julia 

Yeah, that's exactly what that is. I just checked it against the notes I had in Excel for each journal and both those fields are blank.


On Thu, Feb 27, 2014 at 10:33 AM, James Howison <jhowison@ischool.utexas.edu> wrote:
bioj:J_MOL_CATAL_B-ENZYM rdf:type bioj:journal ;
        dc:identifier <urn:issn:1873-3158> ;
        dc:title "Journal of Molecular Catalysis B: Enzymatic" ;
        bioj:strata "111-1455" ;
        bioj:hasAuthorInstructions "http://www.elsevier.com/journals/journal-of-molecular-catalysis-b-enzymatic/1381-1177/guide-for-authors" ;
        bioj:hasSoftwarePolicy "text" ;
        bioj:hasSoftwarePolicy "url" ;
.

Doesn't look quite right.  Are the bottom two just left-over templates and you meant that that policy doesn't include any Software Policy?

Thanks,
James

---------------
Fri Mar  7 22:22:28 MST 2014
---------------

What are queries that actually matter?

What proportion of articles had software mentions?

What is the count of software mentions per article?
Output so that they can be graphed with ggplot2.

---------------
Fri Mar  7 22:27:47 MST 2014
---------------

Note: what do I need to do to move the reference selections from being associated with their in-text location to being directly associated with the articles?

bioj:a2002-39-SYST_BOT-B04 rdf:type bioj:selection ;
	bioj:full_quote "All phylogenies were inferred using the program PAUP* version 4.0b6 (Swofford 2001). Bootstrap analyses based on 1,000 replicates were performed to explore the robustness of the trees. The following options were used for all analyses: parsimony, branch and bound search, stepwise furthest addition sequence, collapse zero-length branches." ;
	bioj:has_reference bioj-cited:Swofford-2001 ;
.

bioj-cited:Swofford-2001 rdf:type bioj:selection ;
	ca:memo "Paup* is available from Sinaeur Associate's online catalogue." ;

	bioj:full_quote "SWOFFORD, D. 2001. PAUP*: Phylogenetic Analysis Using Parsimony, vers.4.0b6. Sunderland, Massachusetts: Sinauer Associates, Inc. Publishers." ;
	
	ca:isTargetOf 
	[ rdf:type ca:CodeApplication ;
	  ca:hasCoder "jbullard" ;
	  ca:appliesCode citec:project_name ;
	] ;

.

if 

?article bioj:has_selection ?sel .
?sel bioj:has_reference ?ref .

then

?article bioj:has_selection ?sel .

no, that doesn't put the codes on the original selection ... which is really the mention.

if 
?article bioj:has_selection ?sel .
?sel bioj:has_reference ?ref .
?ref ca:isTargetOf ?code .

then
?sel ca:isTargetOf ?code .


or, without adding additional statements:

when querying for ?code use

?article bioj:has_selection ?sel .
?sel ca:isTargetOf ?code .
OPTIONAL {
  ?sel bioj:has_reference ?ref .
  ?ref ca:isTargetOf ?code .
}

select * where 
{
	bioj:a2002-39-SYST_BOT bioj:has_selection ?sel .
	?sel ca:isTargetOf [ ca:appliesCode [ rdf:type ?code ] ] .
	OPTIONAL {
  		?sel bioj:has_reference ?ref .
  		?ref ca:isTargetOf [ ca:appliesCode [ rdf:type ?code ] ] .
	}
}


---------------
Mon Mar 10 09:16:58 CDT 2014
---------------


hmm, trouble querying between:

ca:isTargetOf 
[ rdf:type ca:CodeApplication ;
  ca:hasCoder "jbullard" ;
  ca:appliesCode citec:software_used ;
] ;

And
 
ca:isTargetOf 
[ rdf:type ca:CodeApplication ;
  ca:hasCoder "jbullard" ;
  ca:appliesCode [ rdf:type citec:software_name ;
  				   rdfs:label "PAUP*" ] ;
] ;


ca:appliesCode citec:software_used ;  is a code, with a known URL, code name encoded in URL
ca:appliesCode [ rdf:type citec:software_name ;
				   rdfs:label "PAUP*" ] ; is also a code, with an anon URL, code name encoded in type statement for anon node.
				
For consistency, could change all 
ca:appliesCode citec:software_used ;

to 
  ca:appliesCode [ rdf:type citec:software_used ];

Then one has this hierarchy:

citec:software_used rdf:type citec:code .
citec:software_name rdf:type citec:code .

Such that the specific code applied is an anon instance of that code, allowing the addition of memos and labels carrying specific info (e.g., name of the software, version number etc.)

Ask the intuitive question: what type of a code is this? ==> software_name

Would need to:

sed 's/citec:software_used/[ rdf:type citec:software_used ]/g'

query for 
?s ca:appliesCode ?code
FILTER NOT EXISTS {
  ?code rdfs:label ?label
}

---------------
Mon Mar 10 09:32:53 CDT 2014
---------------
			
hmmmm, inconsistent prefixes:

http://james.howison.name/ontologies/citation-codes#in-text_mention

citec http://james.howison.name/ontologies/citation-codes

http://james.howison.name/ontologies/software-citation-coding#in-text_mention

Fixed in https://github.com/jameshowison/softcite/commit/574a264fd9e9a73bb8548fa44fb5d996e24817af

---------------
Mon Mar 10 09:34:36 CDT 2014
---------------

Back to making the citec codes consistent.  Would need to change:

citec:in-text_mention
citec:software_used
citec:identifiable
citec:findable
citec:unfindable_version
citec:access_free
citec:source_unavailable
citec:prohibited_modify
citec:matchesPreferredCite
citec:domain_publication
citec:configuration_details
citec:unidentifiable
citec:unfindable
citec:source_available
citec:permission_modify
citec:software_publication
citec:no_access
citec:users_manual
citec:access_purchase
citec:project_name
citec:missesPreferredCite
citec:project_page
citec:software_not_used
citec:findable_version
citec:users_guide

Or:

ca:isTargetOf 
[ rdf:type ca:CodeApplication ;
  ca:hasCoder "jbullard" ;
  ca:appliesCode [ rdf:type citec:software_used ];
] ;

This seems to work:

sed 's/ca:appliesCode citec:\(.*\)[[:space:]]/ca:appliesCode [ rdf:type citec:\1 ]/g'

What test will help ensure things are not fucked up:

How many code applications:

SELECT COUNT?code
WHERE {
  ca:isTargetOf [ ca:appliesCode ?code ]
}

Also counting codeTypes

For direct-coded this is (ie unlabled)

?s ca:appliesCode ?codeType .
FILTER NOT EXISTS {
  ?codeType rdfs:label ?label
}

For indirect coded this is (ie labeled)

?s ca:appliesCode ?code .
?code rdf:type ?codeType
FILTER EXISTS {
  ?code rdfs:label ?label
}

Thus together:

{ 
?s ca:appliesCode ?codeType .
FILTER NOT EXISTS {
  ?codeType rdfs:label ?label
}
} UNION {
?s ca:appliesCode ?code .
?code rdf:type ?codeType
FILTER EXISTS {
  ?code rdfs:label ?label
}
} 

---------------
Mon Mar 10 10:26:11 CDT 2014
---------------

sed 's/^		\([	]*\)\(.*\)$/		queryStr.append(" \1\2 ");/g'

---------------
Wed Mar 12 10:03:37 CDT 2014
---------------

All articles with in-text_mentions:

select * where 
{
	?article bioj:has_selection ?sel .
	?sel ca:isTargetOf [ ca:appliesCode [ rdf:type citec:in-text_mention ] ] .
	OPTIONAL {
  		?sel bioj:has_reference ?ref .
  		?ref ca:isTargetOf [ ca:appliesCode [ rdf:type citec:in-text_mention ] ] .
	}
}

---------------
Mon Mar 17 08:12:58 CDT 2014
---------------

What are queries that actually matter?

What proportion of articles had software mentions?  By strata?  Get a ggplot2 plot of this?

What is the count of software mentions per article?
Output so that they can be graphed with ggplot2.

---------------
Thu Mar 20 08:59:47 CDT 2014
---------------

What is a general way to turn this from open world to closed world.  ie when I ask for software mentions I get (plyr style)

article, code, value
mentioned, citec:mention, true
unmentioned, citec:mention, false
software_name, citec:software_name, excel
no_name, citec:software_name, false

Could add a code (but that would need a selection?) rdfs:label for each of those that don't have it.

Could make the queries all UNION  or some kind of outer join by using OPTIONAL.

select * where 
{
	?article rdf:type bioj:article .
	OPTIONAL {
		?article bioj:has_selection ?sel .
		?sel ca:isTargetOf [ ca:appliesCode [ rdf:type citec:in-text_mention ] ] .
	}
	OPTIONAL {
  		?sel bioj:has_reference ?ref .
  		?ref ca:isTargetOf [ ca:appliesCode [ rdf:type citec:in-text_mention ] ] .
	}
}

Or could post-process the query against one that has default false

SELECT ?article WHERE
 { ?article rdf:type bioj:article }

then set the code to false for all.

articles$code <- "citec:mention"
articles$value <- false

then rest the code to true just for those that have that code

articles$value

apparently this has to do with the ?with command, or ifelse, but really it's a 

y <- with(DF, (Indoor - Outdoor >= 1) * 1)

---------------
Thu Mar 20 10:43:01 CDT 2014
---------------

New Strategy.  Use SWRL rules.


article(?a), has_selection(?a, ?sel), isTargetOf(?sel,?ca), appliesCode(?ca, ?code), in-text_mention(?code) -> ArticleWithMention(?a)

Article(?a), hasMention(?a, ?m) -> ArticleWithMention(?a)

---------------
Thu Mar 20 19:24:39 CDT 2014
---------------

Ok, new, new strategy, SPIN (or simply CONSTRUCT SPARQL queries).  Switching current changes into branch called nestedOptionals.

basically need to declare and store some spin:rules

http://spinrdf.org/spin.html#spin-rules

According to:

http://topbraid.org/examples/spinsquare.n3

What I need to do is to attach a spin:rule to a Class declaration.  That spin:rule then contains a CONSTRUCT sparql query.  Ideally I can store those in textual format.

Presumably when I then run an ontology model with inferencing the CONSTRUCT query is run.

So what CONSTRUCT query to run.

Want to infer that a bioj:article with at least one selection that has a code in-text_mention applied is a bioj:ArticleWithMention

CONSTRUCT {
  ?this rdf:type bioj:ArticleWithMention
} 
WHERE {
  ?this rdf:type bioj:article ;
        bioj:has_selection ?sel .
  ?sel  ca:isTargetOf [ ca:appliesCode [ rdf:type citec:in-text_mention ] ] .
  OPTIONAL { 
        ?sel bioj:has_reference ?ref .
        ?ref ca:isTargetOf [ ca:appliesCode [ rdf:type ?code ] ] .
  }
}

Ok!  That worked :)

===============================================================
Tue Jun 17 10:23:31 CDT 2014
===============================================================

Trying to recover agreement coding.

Three parts.

1. Raw percent agreement on identifying software 

2. Cohen's Kappa on mention characteristics codes applied by James and Catherine

? Did we retest when training Julia as a coder?

3. Cohen's Kappa on Julia and James searching for software codes



1. Raw agreement.

This was done with the JamesCoding.ttl and CatherineCoding.ttl.  James went through and inspected the fulltext quotes, allocating bioj:matches with the URL of Catherines.  Those without a match were coded bioj:none.  Only CatherineCoding.ttl every ended up in the SoftwareCiationDataset.ttl, because didn't want multiple coders applying codes to selections (too complex for querying.).  That's why there are bioj:matches none statements in the SoftwareCitationDataset.ttl, but none with actual matches.

TODO: move the JamesCoding.ttl and CatherineCoding.ttl files into a new folder in data.
TODO: write rrdf queries to get dataset for percentage agreement (this is rewriting the jena java files)
TODO: write short code for agreement.  Custom code because excluding both disagreed as agreement?  Could still use agree in irr package, I think.

Can test on testingMatchingQueries.ttl.

Ok.  Took JamesCoding.ttl and CatherineCoding.ttl from the CombinedDataset folder.

Hmmm, querying weirdness, moved prefixes from SoftwareCitationDataset.ttl just in case that's the problem.

===============================================================
Wed Jun 18 15:29:44 CDT 2014
===============================================================

Note that I took out a bioj:matches bioj:none for bioj:a2008-39-NAT_BIOTECHNOL-C03, since it was actually referenced by J04.

===============================================================
Thu Jun 19 09:48:51 CDT 2014
===============================================================

Ran the change of non-labeled codes to anon nodes through the agreement files.

sed 's/ca:appliesCode citec:\(.*\)[[:space:]]/ca:appliesCode [ rdf:type citec:\1 ]/g'

===============================================================
Fri Jun 20 15:04:51 CDT 2014
===============================================================
question.  Should there be selections with more than one has_ref?  eg

bioj:a2003-22-CR_BIOL-C02        2
bioj:a2007-36-BMC_PLANT_BIOL-B03        2

Causes havoc with mention typing ... and some other things.  Also throws off mention count ...

Also typing seems to have missed some mentions.

length(unique(types$selection))
[1] 274

There are 284 mentions.  So which ones are missing and why?

Really should put those types on via SPIN rules.

SELECT ?selection ?article ?journal ?strata ?ref
WHERE {
  ?article bioj:has_selection ?selection .
  ?article dc:isPartOf ?journal .
  ?journal bioj:strata ?strata .
  ?selection ca:isTargetOf [ ca:appliesCode [ rdf:type citec:in-text_mention ] ] . 
  ?selection ca:isTargetOf [ ca:appliesCode [ rdf:type citec:software_name ] ] .
  ?selection ca:isTargetOf [ ca:appliesCode [ rdf:type citec:creator ] ] .
  FILTER NOT EXISTS { ?selection bioj:has_reference ?ref }
  FILTER (?selection = bioj:a2009-31-NATURE-C04)
}

hopelessly stuck on getting bioj:a2009-31-NATURE-C04 to show up in the query for like_instrumentation citation type.

Unbelievable.  It was a problem with the testing filter.  can't do ?selection = bioj:a2009-31-NATURE-C04

must do.

FILTER regex(str(?selection) , "a2009-31-NATURE-C04", "i")

Ok, everything has a type except for:

(no_type <- data[!data$selection %in% types$selection, ])

bioj:a2009-35-NAT_BIOTECHNOL-C06  / C-05.  (which at least makes sense with the two less thing!)

Ok, only two have two types.  Huzzah.

               selection num_of_types
1               bioj:a2003-22-CR_BIOL-C02            2
2        bioj:a2007-36-BMC_PLANT_BIOL-B03            2

===============================================================
Mon Jun 23 16:05:38 CDT 2014
===============================================================
Checking to see whether coding softare mentions with multiple citations were coded as multiple selections.


#############
# How many selections have greater than one has_reference?
# clearly bioj:a2003-22-CR_BIOL-C02 and bioj:a2007-36-BMC_PLANT_BIOL-B03 do (both have at least 1 soft pub and 1 domain pub )
#############

query <- "
SELECT ?selection ?ref
WHERE {
  ?selection bioj:has_reference ?ref
}
"

multi_refs <- data.frame(sparql.rdf(softciteData, paste(prefixes, query, collapse=" ")))


multi_refs %.%
group_by(selection) %.%
summarize(num_refs = n_distinct(ref)) %.%
arrange(desc(num_refs))

#################
# How many coded the other way (same full_quote multiple selections, one for each ref?)
###############
query <- "
SELECT ?selection ?quote ?ref
WHERE {
  ?selection bioj:has_reference ?ref ;
             bioj:full_quote ?quote .
}
"

multi_refs <- data.frame(sparql.rdf(softciteData, paste(prefixes, query, collapse=" ")))


multi_refs %.%
group_by(quote) %.%
summarize(num_sels = n_distinct(selection)) %.%
arrange(desc(num_sels))


Nope.  all of the second kind were multiple bits of software.  So there are just a few with multiple cites, but that is fine.

===============================================================
Tue Jun 24 15:04:33 CDT 2014
===============================================================

Ok, normalizing software names.

Used jaro-winkler distance clustering, then manually inspected clusters.

Changes made.

new code in software_name.  bioj:normalized_name
- or just add this as a column in the R analysis?

groups to merge (based on reading clusters)

Image J,ImageJ
Excel,excel
BLAST,Basic Local Alignment Search Tool (BLAST),TBLASTN,BLASTP,BLASTX
SAS,Statistical Analysis Software
Stereo Investigator,StereoInvestigator
USeq,USeq IntersectLists,USeq IntersectRegions,USeq AggregatePlots,USeq FNG
PAUP*,PAUP
ClustalW/ClustalX,ClustalW,CLUSTAL W,CLUSTALW,ClustalX


===============================================================
Tue Jun 24 15:16:49 CDT 2014
===============================================================
a2008-06-NAT_GENET-C02
a2008-06-NAT_GENET-C01

===============================================================
Wed Jun 25 13:35:44 CDT 2014
===============================================================

Ug. there are only 7 unidentifiable.  Thus the 156 / 211 ratio is definitely wrong.  Need to sort out the bioj:references_same_software and move the codes?
Also Image J is unidentifiable? -- hmmm, ImageJ preferred over Image J, fixed in standardized names.  But how has that affected the analysis of the codes?

I think I need to re-organize the codes.  

For identifiable/findable/findable version those should be for software-article.

article contains article_software_link (merger of all mentions in that article)
	software_mention contributes to article_software_link.
	

Declarative style:

Create a mention by merging all of the codes from references.

Create a ReferenceSelection type for any selection that is the RHS of a has_reference statement.
Create a InTextSelection type for any selection that has the "in-text_mention" code.


Create a new URI for each InTextSelection as SoftwareMention

Move all of the codes for each selection and any reference onto that SoftwareMention.
Include links back to the Selections

Using standardized_names, create a SoftwarePackage URI, one for each standardized Name.

For each Article
  For each SoftwarePackage
  	if SoftwarePackage in the article

Create SoftwareInArticleLink
link to each of the SoftwareMention that map to it using the software_name <-> standardized_name link.
link to the SoftwarePackage through standardized_name -> software_name link.
Add attributes according to tags on SoftwareMention
coded
identifiable ->  is_identifiable TRUE / FALSE
findable -> is_findable TRUE / FALSE
creator -> is_credited TRUE / FALSE

Add attributes to SoftwarePackages, according to tags on linked SoftwareInArticleLink -> Selections
if any Selection is coded 

acesss_purchase -> is_accessible = T, is_free = F
access_free -> is_accessible = T, is_free = T
no_access -> is_accessible = F

source_available -> is_source_available = T / F
permission_modify -> is_explicitly_modifiable = T /F

===============================================================
Wed Jul  2 15:23:57 CDT 2014
===============================================================

 
Move typing from R analysis to SPIN rules.  Should be pretty simple combination of true and falses.

===============================================================
Wed Jul  2 16:27:14 CDT 2014
===============================================================

Discovered, in a2000-42-NATURE-C05 that the software has 2 names, one in the text, one in the reference.  Merging those through the name mapping from CNS to 

Hmmm, this is because mentions end up with multiple original_names.

Change alterntive strategy.
# replace alternatives with standard
INSERT {
  ?s ca:appliesCode [ rdf:type citec:software_name ;
    			      rdfs:label "CNS" ]
}
WHERE {
  ?s ca:appliesCode [ rdf:type citec:software_name ;
    				  rdfs:label "Crystallography & NMR system" ]
}

# delete alternatives
DELETE {
  ?s ca:appliesCode ?ca
}
WHERE {
  ?ca rdf:type citec:software_name ;
      rdfs:label "Crystallography & NMR system" 
}

Not clear I can do this right now.

Going to replicate types analysis via SPIN.

===============================================================
Mon Jul 28 09:25:04 CDT 2014
===============================================================

Seems I have done the types replication via SPIN?

There was some confusion with multiple runs of the SPIN rules?  I have one set that runs, then some Java stuff happens about project names, then another set runs.

Hmmmm, the name mappings aren't working perfectly.  Not sure if I should add the ones with alternatives, or let the SPIN rules do that.  Checking now.

Ok, seems that it was a missing inverse statement. Fixed.


a2003-44-SCIENCE-BLASTX.  Should that exist? It's an alternative name, no? It's all good, that's an ArticleSoftwareLink correctly linked to citec:software-BLAST.

===============================================================
Mon Jul 28 10:14:26 CDT 2014
===============================================================

Hmmm, some are missing is_findable true/false.  e.g., bioj:a2004-46-NATURE-GAP

<http://james.howison.name/ontologies/bio-journal-sample#a2007-14-NAT_GENET-Image%20J>

bioj:a2003-44-SCIENCE-TBLASTN


Seems this is related to bioj:references_same_software, since those codes are created by tracking from the Mention back to the Selection and looking for codes there.

Could I create a SPIN validation to discover the missing is_findable ?  Probably.

===============================================================
Mon Jul 28 16:48:13 CDT 2014
===============================================================

Ok, I think the referencesSameSoftware was a red herring, because those tags on one of the mentions will bring the properties to the ArticleSoftwareLinks.

a2008-06-NAT_GENET-C02 -> DatLab just doesn't have the coding.  And it is software_used.

Also a2007-51-NAT_GENET-C01 "EIGENSTRAT"

a2007-14-NAT_GENET-C04

So far, these have all been Catherine Coded, perhaps Julia just didn't do these?  No, a2003-39-J_COMPUT_AID_MOL_DES-B01 is a Julia coded one.

===============================================================
Wed Aug 13 16:33:10 CDT 2014
===============================================================

bioj:a2003-39-J_COMPUT_AID_MOL_DES-CScore - this is software_not_used.

Note that these don't have identifiable or findable, but they do have credited true because that one is done by negation, a pretty bad idea, really.

===============================================================
Fri Aug  1 08:45:09 CDT 2014
===============================================================

Hey, SPIN is cool, I was very glad to find it.  I've messed around with SWRL a while back, but given that I vaguely understand SPARQL this is much, much, easier.
 
If you do edit these files by hand, I would still recommend to make them easier to open with TopBraid (Free Edition) and similar RDF tools. All you would need to do is add some owl:Ontology into every file and define explicit owl:imports so that when you load the example file, it will pull all other necessary files into TopBraid. This may help you get more feedback because more people could understand what's going on (without stepping through Java code). Just an idea.

I've created a new branch for my attempt to do this:

https://github.com/jameshowison/softcite/tree/topbraid

It's a little complicated by the need to "drop out" to some Java code to do some name mapping (some of the programs that I'm writing data about have alternative names), quite likely this can be done via SPIN but it isn't at the moment.

If I understand correctly, I should have a "main file" of rdf that imports the other files.  Would one typically do a vocab file as the main file, then import the individuals and the spin rules and constraints?  At present I've just add the owl imports for SPIN etc and opened my SPINConstraints.ttl file in TopBraid FE. In the SPIN overview tab I see only the constraint for bioj:ArticleSoftwareLink, but not the constraints for 

===============================================================
Fri Aug 15 09:25:38 CDT 2014
===============================================================

These appear to lack the misses or matches coding.

bioj:a2002-39-SYST_BOT-B04

bioj:a2005-46-NATURE-C11 / bioj:a2005-46-NATURE-C08

bioj:a2005-46-NATURE-C01

Ok, these are to packages that don't have a preferredCite.

===============================================================
Fri Aug 15 10:58:04 CDT 2014
===============================================================

bioj:software-Opticon%20Monitor%202  

===============================================================
Thu Jan  8 12:40:05 CST 2015
===============================================================

are people consistent within an article.

What if you treat it by article?  but not consistent between articles - what matters for the question of diversity is what readers perceive, and since they read articles by many authors, the data shows that they'd see a large variety of types of mentions.'  The article/journal level influence makes it appear like there would be more consistency, so it's a conservative hypothesis.'


===============================================================
Thu Jan  8 12:54:31 CST 2015
===============================================================

CrossTable in gmodels, sresid=T.  g>2 or <-2.  posthoc for Chisq test.

===============================================================
Fri Jan  9 09:34:41 CST 2015
===============================================================

- consider a formal hypothesis test for the between strata parts that actually seem to be statistically significantly different.
1. redo Fig 1 without the zero articles.
2. do comparison between commercial and non-commercial software, does it determine "like instrumment" vs others?
- redo abstract to include plus/minus on percentages.