Skip to content

Commit f6a2092

Browse files
committed
Add 'CONSIDERATIONS.md' document
This hopefully helps with making more informed decisions about when to use large objects instead of some other storage mechanism.
1 parent aeae518 commit f6a2092

3 files changed

Lines changed: 83 additions & 1 deletion

File tree

CHANGELOG.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,9 @@
11
# CHANGELOG
22

3+
## v0.2.1 - ?
4+
5+
* Add 'Considerations' document
6+
37
## v0.2.0 - 2025-11-13
48

59
* Add `PgLargeObjects.UploadWriter`

CONSIDERATIONS.md

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
# Considerations
2+
3+
The [PostgreSQL 7.1 documentation](https://www.postgresql.org/docs/7.1/largeobjects.html) explains:
4+
5+
> Originally, Postgres 4.2 supported three standard implementations of large objects
6+
7+
PostgreSQL 4.2 was released on Jun 30th, 1994. The large objects facility has
8+
been around for a long time!
9+
10+
Yet, it is fairly unknown to many programmers - or considered too unwieldy to
11+
use for productive usage. This is not by accident - there are various trade
12+
offs to consider when deciding if large objects are a good mechanism for
13+
storing large amounts of binary data.
14+
15+
This document attempts to collect and discuss some of these considerations. If
16+
you feel there are other aspects to highlight, or if any of the items below
17+
warrants further elaboration, please don't hesitate to submit a GitHub pull
18+
request!
19+
20+
## Partial vs. Complete Data
21+
22+
The pg_large_objects library, at its highest level, exposes large objects as data
23+
streams by defining appropriate implementations of the Enumerable (for reading)
24+
and Collectable (for writing) behaviour. This is only possible by taking advantage
25+
of the fact that large objects enable working with *partial* data. Objects can be
26+
read and written in small chunks, and operations like
27+
`PgLargeObjects.LargeObject.seek/2` enable accessing individual parts of a large
28+
object without loading the entire data into memory.
29+
30+
If your application always only needs to work with the entire data as a whole,
31+
and loading it into memory as a whole is possible and convenient, the
32+
application might be better off with storing the data in a `bytea` column of
33+
a table.
34+
35+
## Storage Costs
36+
37+
Storing large objects in a PostgreSQL database may greatly increase the amount
38+
of disk space used by the database. This may be more expensive than other
39+
mechanisms for storing large objects.
40+
41+
For example, the [AWS RDS
42+
documentation](https://aws.amazon.com/rds/postgresql/pricing/) (RDS is Amazon's
43+
managed database offering) explains that at the time of this writing, 1GB of
44+
General Purpose storage for a in the us-east-1 region costs $0.115 per month
45+
for a PostgreSQL database. The [AWS S3 documentation](https://aws.amazon.com/s3/pricing/) (S3 is Amazon's
46+
object storage offering) documents, at the time of this writing, that storing
47+
1GB of data in the us-east-1 region is a mere $0.023 per month!
48+
49+
I.e. when using Amazon cloud services in the us-east-1 region, storing data in
50+
RDS is five times as expensive as storing it in S3. Depending on the amount of
51+
data and your budget, this might be a significant difference.
52+
53+
Make sure to check the pricing (if applicable) for storage used by your
54+
PostgreSQL database and consider the change in the decision whether to use
55+
large objects or not.
56+
57+
## Backups
58+
59+
Given that large objects may quickly end up being the bulk of data stored in a
60+
database, it's common to configure backups to exclude them from backups or only
61+
include them in weekly backups or similar.
62+
63+
For example, the `pg_dump` command line utility features four related options:
64+
65+
```
66+
frerich@Mac ~ % pg_dump --help
67+
[..]
68+
-b, --large-objects include large objects in dump
69+
--blobs (same as --large-objects, deprecated)
70+
-B, --no-large-objects exclude large objects in dump
71+
--no-blobs (same as --no-large-objects, deprecated)
72+
[..]
73+
```
74+
75+
Consider your current backup mechanism and see if it's configured to include or
76+
exclude large objects. Decide on the important of large objects for your use
77+
case and include that in your decision on how often large objects should be
78+
included in backups.

mix.exs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ defmodule PgLargeObjects.MixProject do
2424
source_url: "https://github.com/frerich/pg_large_objects",
2525
docs: [
2626
main: "readme",
27-
extras: ["README.md", "cheatsheet.cheatmd", "CHANGELOG.md", "LICENSE.md"]
27+
extras: ["README.md", "cheatsheet.cheatmd", "CONSIDERATIONS.md", "CHANGELOG.md", "LICENSE.md"]
2828
]
2929
]
3030
end

0 commit comments

Comments
 (0)