-
Notifications
You must be signed in to change notification settings - Fork 92
Filecoin and IPFS interactions
A quick summary of our understanding of how IPFS and Filecoin relate to each other.
Libp2p provides most of the IPFS functionality:
- a connected mesh of peers
- content-addressed routing (via the DHT)
- a protocol to copy files around (bitswap)
libp2p let's us multiplex protocols on the wire so peers can communicate to each other using any of the protocols they have agreed upon to understand.
So, when we say IPFS what we mean is libp2p using the bitswap protocol plus a blockstore
which is IPLD content saved to disk in sectors.
Filecoin also uses libp2p but on a distinctly different network of peers. It also extends the list of protocols to use graphSwap (as well as bitSwap) plus a filecoin specific protocol.
OPEN QUESTION: do both IPFS and Filecoin save CAR files to disk
OPEN QUESTION: is an IPFS repo (i.e. on disk storage) compatible with Filecoin?
The following things are separate but related in the following way:
- The IPFS codebase
-
The Filecoin codebase e.g. Lotus
- “all Filecoin nodes are IPFS nodes under the hood”) <---- clarification they both use libp2p and bitswap under the hood rather than filecoin uses IPFS under the hood. Filecoin does not depend on an IPFS node running locally rather it shares the same codebase via libp2p
-
The IPFS network <- a standalone network for storage but with no incentive structure for persistence (you just make data you’ve got locally available on IPFS by publishing it, others can download it, but if you go offline, that data isn’t accessible any more - "Filecoin and IPFS are two separate, complementary protocols")
- This network is specified/seeded by a bootstrap list
-
The Filecoin network <- separate networks (“Filecoin nodes don’t join or participate in the public IPFS DHT”, “Filecoin is the missing incentive layer for IPFS and verifies that your data is being stored, while maintaining the efficiency, authenticity and resiliency provided by IPFS”)
- The filecoin network is not a seperate instance of the IPFS network
I do not believe that you can actually “ignore IPFS completely” for what you intend to do
Unless you will design a side-protocol to distributed the computations to be run
- The above means "how are you going to transport the compute to the node that will run it?"
- You can use IPFS
- You can write your own protocol
- (lets just use IPFS)
The thinking here is that:
- IPFS is composed of libp2p+IPLD+small amount of extra code that is ipfs specific
- You will use libp2p to connect with Filecoin nodes, which is the largest chunk of IPFS codebase
- IPLD https://ipld.io/ is what IPFS files and Filecoin Car files are made of, and you can use it to describe versions of datasets that were changed or even the computation itself. Check https://research.protocol.ai/publications/ipfs-fan-a-function-addressable-computation-network/delarocha2021a.pdf for inspiration
QUESTION - does Lotus run IPFS in-process? The Lotus install docs don't specify needing to separately install an IPFS node, but the code looks like it's assuming an IPFS server is (can be) running locally.
ANSWER - No - Lotus uses libp2p and bitswap which is the same as ipfs but it does not run the ipfs server under the hood
QUESTION - is the filecoin network two layered networks (IPFS + blockchain on top) or is it all one libp2p mesh?
ANSWER - No - it is a single network with 3 multiplexed protocols (because libp2p lets you do that (neat)) - bitswap, graphswap and filecoin specific protocol (streaming protocol to update the chain)
OPEN QUESTION - when specifying, say, a list of CIDs for a Bacalhau job to run on, would we expect to be able to specify CIDs in the IPFS network, or the Filecoin network? In the latter case, they would be sealed, and need to be unsealed. Assuming we specify the CIDs as IPLD addresses, can IPLD reference files in the Filecoin network?
NOTES - look at the IPFS codebase and find out how to check if a CID lives locally in the repository - do the same with the filecoin codebase and confirm that the format of the CID is the same. Put another way - are CID's truely content addressable and transferable between both networks?
ANSWER - Yes - CID's are the content addressable ids for CAR files (Content Addressable Archives) and these are used both by IPFS and Filecoin. So - it is reasonable to say:
- hey - I have a CID string, is there a node on IPFS that has the contents of the CAR file for this CID?
- no - OK - is there a node on Filecoin that has the contents of the CAR file for this CID?
- https://lotus.filecoin.io/docs/developers/retrieve-data/
OPEN QUESTION - out of curiosity, how do the Filecoin and IPFS networks avoid interfering with eachother, by, say, a node that connects to both at the same time by accident?
NOTES - It's not designed to work this way. IPFS and Filecoin are distinct networks from each other - although they both use libp2p so it is theoretically possible e.g. for a Lotus node to connect to the IPFS network but it would have to speak the right layered protocols in order to interact with it.
OPEN QUESTION - what does Filecoin actually store in its IPFS network? Is it sealed sectors, or something else? Is IPFS used as a way to specify data that is an input to a Filecoin deal?
ANSWER - Filecoin doesn't actually have it's own IPFS network - it just communicates with other nodes over libp2p in the same way. It stores data in directories that the daemon is responsible for reading and writing to. In other words Lotus is stateful and it's a proxy onto some IPFS thing.
--