GlusterFS & SeaweedFS

Created on 2023-03-06T04:33:34-06:00

Return to the Index

This card can also be read via Gemini.

SeaweedFS is an implementation of Facebook's Haystack; it excels in storing a large number of small files or caching access to objects stored in remote paid services. You can for example proxy requests to files to Seaweed which downloads from Google Drive and keeps it in a volume for some period of time to reduce API prices. Does not handle temporary files well though; temp data has to be kept in temp volumes (deleted all at once) or will just be tombstoned within volume indices until separate compaction runs are made explicitly.

GlusterFS stores files primarily. Apparently sensitive to the underlying filesystem tuning as well. Easy to set up and run but doesn't handle tons of tiny files supposedly.