f4: Facebook’s Warm BLOB Storage System
Created on 2021-11-17T22:09:08-06:00
A replacement for Haystack that handles data which is not accessed as frequently.
Relies on Reed Solomon(10,4) and XOR coding instead of simply keeping multiple copies of the file around.
Instead of a directory service the blob information is kept in a graph database.
Accessing data goes through the web frontend / CDN which can do an immediate cache, then internally requesets from blob storage (placing in to hot cache on the way.)
Storage is similar to haystack
Stores files similarly to haystack with physical and logical volumes.
Volumes can be locked or unlocked. A locked volume may no longer be written to.
Journal files now exist which can register deletes in locked volumes.
Temperatures of data
Hot: actively requested data; ex. front page images
Warm: data which is accessed recently but not hot
Cold: stored in a manner which requires time and special access to retrieve