|
| 1 | +# Considerations |
| 2 | + |
| 3 | +The [PostgreSQL 7.1 documentation](https://www.postgresql.org/docs/7.1/largeobjects.html) explains: |
| 4 | + |
| 5 | +> Originally, Postgres 4.2 supported three standard implementations of large objects |
| 6 | +
|
| 7 | +PostgreSQL 4.2 was released on Jun 30th, 1994. The large objects facility has |
| 8 | +been around for a long time! |
| 9 | + |
| 10 | +Yet, it is fairly unknown to many programmers - or considered too unwieldy to |
| 11 | +use for productive usage. This is not by accident - there are various trade |
| 12 | +offs to consider when deciding if large objects are a good mechanism for |
| 13 | +storing large amounts of binary data. |
| 14 | + |
| 15 | +This document attempts to collect and discuss some of these considerations. If |
| 16 | +you feel there are other aspects to highlight, or if any of the items below |
| 17 | +warrants further elaboration, please don't hesitate to submit a GitHub pull |
| 18 | +request! |
| 19 | + |
| 20 | +## Partial vs. Complete Data |
| 21 | + |
| 22 | +The pg_large_objects library, at its highest level, exposes large objects as data |
| 23 | +streams by defining appropriate implementations of the Enumerable (for reading) |
| 24 | +and Collectable (for writing) behaviour. This is only possible by taking advantage |
| 25 | +of the fact that large objects enable working with *partial* data. Objects can be |
| 26 | +read and written in small chunks, and operations like |
| 27 | +`PgLargeObjects.LargeObject.seek/2` enable accessing individual parts of a large |
| 28 | +object without loading the entire data into memory. |
| 29 | + |
| 30 | +If your application always only needs to work with the entire data as a whole, |
| 31 | +and loading it into memory as a whole is possible and convenient, the |
| 32 | +application might be better off with storing the data in a `bytea` column of |
| 33 | +a table. |
| 34 | + |
| 35 | +## Storage Costs |
| 36 | + |
| 37 | +Storing large objects in a PostgreSQL database may greatly increase the amount |
| 38 | +of disk space used by the database. This may be more expensive than other |
| 39 | +mechanisms for storing large objects. |
| 40 | + |
| 41 | +For example, the [AWS RDS |
| 42 | +documentation](https://aws.amazon.com/rds/postgresql/pricing/) (RDS is Amazon's |
| 43 | +managed database offering) explains that at the time of this writing, 1GB of |
| 44 | +General Purpose storage for a in the us-east-1 region costs $0.115 per month |
| 45 | +for a PostgreSQL database. The [AWS S3 documentation](https://aws.amazon.com/s3/pricing/) (S3 is Amazon's |
| 46 | +object storage offering) documents, at the time of this writing, that storing |
| 47 | +1GB of data in the us-east-1 region is a mere $0.023 per month! |
| 48 | + |
| 49 | +I.e. when using Amazon cloud services in the us-east-1 region, storing data in |
| 50 | +RDS is five times as expensive as storing it in S3. Depending on the amount of |
| 51 | +data and your budget, this might be a significant difference. |
| 52 | + |
| 53 | +Make sure to check the pricing (if applicable) for storage used by your |
| 54 | +PostgreSQL database and consider the change in the decision whether to use |
| 55 | +large objects or not. |
| 56 | + |
| 57 | +## Backups |
| 58 | + |
| 59 | +Given that large objects may quickly end up being the bulk of data stored in a |
| 60 | +database, it's common to configure backups to exclude them from backups or only |
| 61 | +include them in weekly backups or similar. |
| 62 | + |
| 63 | +For example, the `pg_dump` command line utility features four related options: |
| 64 | + |
| 65 | +``` |
| 66 | +frerich@Mac ~ % pg_dump --help |
| 67 | +[..] |
| 68 | + -b, --large-objects include large objects in dump |
| 69 | + --blobs (same as --large-objects, deprecated) |
| 70 | + -B, --no-large-objects exclude large objects in dump |
| 71 | + --no-blobs (same as --no-large-objects, deprecated) |
| 72 | +[..] |
| 73 | +``` |
| 74 | + |
| 75 | +Consider your current backup mechanism and see if it's configured to include or |
| 76 | +exclude large objects. Decide on the important of large objects for your use |
| 77 | +case and include that in your decision on how often large objects should be |
| 78 | +included in backups. |
0 commit comments