[0.11.1] - 2022-01-06

Added

• quiet flag for genomepy.Annotation
• genomepy -v flag

Changed

• genomepy.Annotation returns a FileNotFoundError instead of a ValueError where appropriate.
• download_assembly_report refactored. Now downloads the report for the exact same assembly accession (and not the nearest NCBI assembly).
• broader unit tests for UCSC assembly accession scraping

Fixed

• inconsistent behaviour with assembly reports (#193 + #194)

[0.11.0] - 2021-11-18

Added

• extened docstrings
• GENCODE support (GENCODE gene annotations with UCSC genomes)
  • only contains the main chromosomes, no scaffolds or alternate haplotypes.
  • only contains 4 assemblies (2 mouse, 2 human)
  • excellent annotations for these regions & species though!
• Ensembl's GRCh37 can now be downloaded through genomepy
• Local fasta/gtf/gff(3)/bed file support
  • you can install a local genome and/or annotation by providing local path(s) to genomepy install
    • if annotation downloading is requested, but not annotation path is provided,
      a gtf/gff(3) annotation will be sought in the genome's source directory.
• Annotation.gtf_dict creates a dictionary for any key-value pair in the GTF columns or attribute fields!
  • e.g. Annotation.gtf_dict("seqname", "gene_name")

Changed

• Genome.track2fasta can now ignore comment lines (starting with #)
• Genome.track2fasta will skip header lines (a warning will be printed)
• Genome.track2fasta will ignore regions that cannot be parsed (a warning will be printed)
  • these fixes should improve gimme scan performance and feedback
• UCSC annotation conversion tool settings tweaked. Better results with source gff files.
• Ensembl now uses HTTP instead of FTP (in some cases). This improves stability on some servers.
• tweaked search result alignment for clarity
• explained UCSC annotations in the README
• better file path handling (relative paths, user home and variables are expanded)
• Annotation now accepts a file/directory/genomepy name as first argument.
  • this merges 2 arguments into one.
• Annotation.map_genes now works without a README file
  • you can now set Annotation.tax_id manually.

Fixed

• Ensembl annotations from previous releases can now be downloaded as intended.
• Genome.track2fasta will skip regions that clearly dont make sense (start>end, and start<0)

Version 0.10.0

[0.10.0] - 2021-07-30

Added

• Annotation class, containing
  • regex filter (genomepy.Annotation.filter_regex())
  • sanitize functions (genomepy.Annotation.sanitize())
    • option to skip filtering and/or matching the annotation to the genome (also on CLI)
  • gene name remapping to various formats (genomepy.Annotation.map_genes())
    • using MyGene.info. Can be queried separately (genomepy.annotation.query_mygene())
  • contig name remapping to other provider formats (genomepy.Annotation.map_locations())
  • get the annotations, or gene locations, as dataframes (genomepy.Annotation.gtf, bed or gene_coords() respectively)
  • get the gene names as a list (genomepy.Annotation.genes("gtf") or genomepy.Annotation.genes("bed"))
• genomepy install now attempts to install the NCBI assembly report
• NCBI provider also indexes the NCBI genbank_historical summary
• genomepy search now shows if the genome has an annotation
  • this slows down the results a bit
  • to compensate, results are now shown as soon as they are found
  • for UCSC, availability of any of the 4 annotations is shown
• genomepy annotation shows the first line(s) of each gene annotation.gtf
• for developers:
  • pre-commit-hooks for linting
  • formatting/linting script tests/format.sh (optional argument lint)
  • isort & autoflake formatters

Changed

• provider module split per provider
• ProviderBase overhauled, now called Provider
• regex filtering separated from Provider.download_genome
• utils module split into utils, files and online
• now using loguru for pretty logging
• accession search improved
  • now finds GCA and GCF accessions
  • now ignores patch levels
• genomepy install automatic provider selection refactored
  • Provider.online_providers returns a generator (faster!)
• genomepy install uses a combined filter function (faster!)
• genomepy install only zips annotation files if the genome is zipped (with the bgzip flag) (faster!)
• NCBI provider should be parsed faster (faster!)
• new dependency: pandas
• tests no longer format code

Fixed

• broken URLs should keep genomepy occupied for less long (check_url will immediately return on "Not Found" errors 404/450) (faster!)
• the Genome class now passes arguments to the parent Fasta class
• the Genome class now regenerates the sizes and gaps files similarly to the Fasta class and its index (when the genome is younger) (faster!)
• somewhat more pythonic tests

Version 0.9.3

[0.9.3] - 2021-02-03

Changed

• URL provider got better at searching for annotation files
• NCBI provider will fall back on FTP if HTTPS is offline

Fixed

• genomes from ftp locations not working

Version 0.9.2

[0.9.2] - 2021-01-28

Added

• progress bars for downloading and bgzipping (the slow stuff)
• spinner to indexing plugins (the slowest stuff)

Changed

• removed dependency of psutils
• added dependency of tqdm

Fixed

• an oopsie in the regex filter functions slowing down install.
• rm_rf and mkdir_p to behave more like their namesakes.

Version 0.9.1

[0.9.1] - 2020-10-26

Added

• genomepy install flag -k/--keep-alt to keep alternative regions
• argparse custom type for a genome command line argument

Changed

• added retries to UCSC and NCBI
• added retries to Travis tests
• Bucketcache improvements
• genomepy search keeps searching after an exact match is found
• genomepy install removes alternative regions by default

Fixed

• genomepy clean wont complain when there is nothing to clean
• properly gzip the annotation.gtf if it was unzipped during sanitizing
• genomepy install can use the URL provider again
• genomepy install with -f/--force will overwrite previouse sizes and gaps files

Version 0.9.0

[0.9.0] - 2020-09-01

Added

• check to see if providers are online + error message if not
• automatic provider selection for genomepy install
  • optional provider flag for genomepy install (-p/--provider)
  • if no provider is passed to genomepy install, the first provider with the genome is used (order: Ensembl > UCSC >
    NCBI).
• genomepy clean removes local caches. Will be reloaded when required.

Changed

• Ensembl genomes always download over ftp (http was too unstable)
• Ensembl release versions obtained via REST API (http was too unstable)
• genomepy search and genomepy providers only check online providers
• Online function now have a timeout and a retry system
• API changes to download_genome and download_annotation for consistency

Fixed

• Ensembl status check uses lighter url (more stable)
• search and install now consistently use safe search terms (no spaces)
• search now uses UTF-8, no longer crashing for \u2019 (some quotation mark).
• search case insensitivity fixed for assembly names.
• Bucketcache now stores less data, increasing responsiveness.

Version 0.8.4

[0.8.4] - 2020-07-29

• Fix bug where Genome.sizes dict contains str instead of int (#110).
• Fix bug with UTF-8 in README (#109).
• Fix bug where BED files with chr:start-end in 4th column are not recognized as BED files.

Version 0.8.3

[0.8.3] - 2020-06-03

Fixed

• Fixed bug introduced by fixing a bug: Provider-specific options for genomepy install on command line work again
• UCSC annotations can now once again be obtained from knownGene.txt

Added

• UCSC gene annotations will now be downloaded in GTF format where possible
• Desired UCSC gene annotation type can now be specified in the genomepy install command using --ucsc-annotation

Changed

• Added the NCBI RefSeq gene annotation to the list of potential UCSC gene annotations for download

Version 0.8.2

[0.8.2] - 2020-05-25

Fixed

• Genome.sizes and Genome.gaps are now populated automatically.
• backwards compatibility with old configuration files (with genome_dir instead of genomes_dir)
• updating the README.txt will only happen if you have write permission
• after gzipping files the original unzipped file is now properly removed
• providers will only download genome summaries when specifically queried

Changed

• updated blacklist for hg38/GRCh38 based on work by Anshul Kundaje, see ENCODE README.txt