Missing placements examine assign

[basic99]

Hi,

We find a descrepancy with the number of queries placed on the tree with gappa examine graft and gappa examine assign. I attach the jplace file and the per_query.tsv. The tree has 3827 placements which is the same number that are in the jplace file. However, the placements report created with gappa examine assign only assigns 3465 placements. In other words about 10% of the placements have gone missing.

/usr/local/gappa/bin/gappa examine assign --jplace-path epang/epa_result.jplace --taxon-file taxon_file.tsv --out-dir assign --per-query-results

epa_result.jplace.txt
per_query.tsv

Jim White

[lczech]

Hi Jim @basic99,

that is interesting. As the assign command is the only one I did not develop myself in gappa, I don't have an answer for you already, and would need to look into this. Could you also please share the taxon_file.tsv file here, such that I can try to reproduce your issue?

My current best guess based on the information and data that you provided is that these missing placements are those that do not fall exactly within one taxonomic group, and are hence filtered out, see here. For instance, visualizing the placement of your QUERY___OTU2254, which is one of the missing ones, it has three nearby placement locations:

Depending on your taxon file, these might or might not be in the same taxonomic clade. This could be the cause - maybe have a look if that makes sense. And if not, share the taxon file here, and I can also investigate a bit.

Cheers and so long
Lucas

[basic99]

Hi,
I am attaching 2 files. The simple one with just the taxon name is what we had been using with our tool. It places 3465 out of 3827 tree tips. I have made another taxon file which I think is closer to what the tool expects for input and placed 3799, although I think some of these placements don't place a taxon. We have not yet decided how to apply this to our results.

taxon_file.csv

taxon_file2.csv

Thanks,
Jim W

[lczech]

Hi @basic99,

thanks for you patience, I was traveling, and just got the chance to have another look at this.

With your taxon file, I could now inspect those files, and re-run the command. The issue is that your taxonomy is not really hierarchical, and instead just lists taxon names without resolving their taxonomic tree. You already hinted at this with the second taxon file you provided. However, without this information, there is no way to resolve the taxonomic assignment of anything that isn't placed on a terminal branch.

You can for instance see this when plotting your data via Krona (add the --krona flag to the command):

There is no hierarchy of the taxonomy there. Thus, all placements that are not on terminal branches (leading to a leaf of the phylogenetic tree) cannot be assigned to any taxon.

With your second taxon file, this is somewhat solved:

However, from a brief inspection of the clade of the unassigned query that I visualized in my previous response, it still seems that there are taxon assignments that contradict the tree, and hence lead to clades where placements will not get a taxonomic assignment. For instance, the four tips of the branches where the query in my previous response is placed are:

Clostridium_pabulibutyricum_strain_MJC39 Bacillota;Clostridiales;Eubacteriales;Clostridiaceae;Clostridium;pabulibutyricum
Clostridium_acetireducens_strain_30A     Bacillota;Clostridiales;Eubacteriales;Clostridiaceae;Clostridium;acetireducens
Clostridium_cadaveris_strain_JCM_1392    Bacillota;Clostridiales;Eubacteriales;Clostridiaceae;Clostridium;cadaveris 
Arthromitus_sp                           Bacillota;Clostridia;Eubacteriales;Clostridiaceae;Candidatus arthromitus;sp

As you can see, the last one is labelled as Clostridia, while the first three are labelled as Clostridiales instead. As gappa is not magic, it does not know that these are meant to represent the same taxa (I assume?), and sees them as different. Hence, the query has no clear taxonomic assignment, and is left out from the result.

So, to fix this, I suggest to clean up your taxon file with taxonomic labels that are consistent with the phylogenetic reference tree. Any incongruencies will lead to branches without clear taxonomic assignment, and hence to missing queries placed on those branches.

If the issue still persists then, please let me know :-)

Hope that helps, and so long
Lucas

[basic99]

Hi,
Thanks for the answer. That is an interesting graph. Some of the metadata for the newick files is not correct, so we can look into that. For the missed we have a script that makes a call from the newick file. It does not give a gappa LWR value but can use it in the assignments report.
Jim W

[lczech]

Hi Jim @basic99,

Some of the metadata for the newick files is not correct, so we can look into that.

Sounds good. Maybe to elaborate a bit on my previous answer: The idea of using phylogenetic placement as a means to get taxonomic assignment of the reads only makes sense if the tree and the taxonomy are congruent. That is, clades in the tree are assumed to correspond to clades in the taxonomy. A clade in the tree is any complete set of branches that is separated from the rest of the tree by a single branch. In other words, an inner branch of the tree defines a clade consisting of all the branches split from the tree by the inner branch. For instance using my previous example:

The black branch defines a clade consisting of the two purple and the four blue branches left of it.

These clades should then correspond to clades of your taxonomy, where higher ranks correspond to the inner branches, and lower ranks to the branches towards the outside of the tree, thus reflecting the hierarchy. Any deviation from this (any incongruence between phylogeny and taxonomy) will lead to branches that do not have a clear taxonomy label (or have a rather high, unspecific, label), which is what you have been observing.

For the missed we have a script that makes a call from the newick file. It does not give a gappa LWR value but can use it in the assignments report.

I don't understand what you mean by that. What does that script do?

Cheers and so long
Lucas

[basic99]

Hi,
We make an assignments report of placements on the tree. gappa examine does this but we need to make assignments that are not in per_query file. Since gappa examine graft places all the queries we have a script that picks an arbitrary reference in the same clade as query.
Jim W

[lczech]

Ah okay, I see. Hm yes, could work as a workaround. But the cleaner solution is to use a taxonomy that actually reflects the hierarchical structure of the tree.

Let me know what you find, or if you have any further questions or troubles. We can keep this issue open in the meantime.

Cheers
Lucas