Module forest_filecoin::lotus_json

source ·
Expand description

In the Filecoin ecosystem, there are TWO different ways to present a domain object:

  • CBOR (defined in fvm_ipld_encoding). This is the wire format.
  • JSON (see serde_json). This is used in e.g RPC code, or in lotus printouts

We care about compatibility with lotus/the Filecoin ecosystem for both. This module defines traits and types for handling both.

§Terminology and background

  • A “domain object” is the concept of an object. E.g "a CID with version = 1, codec = 0, and a multihash which is all zero" (This happens to be the default CID).
  • The “in memory” representation is how (rust) lays that out in memory. See the definition of struct Cid { .. }.
  • The “lotus JSON” is how lotus, the reference Filecoin implementation, displays that object in JSON.
    { "/": "baeaaaaa" }
  • The “lotus CBOR” is how lotus represents that object on the wire. 
    let in_memory = ::cid::Cid::default();
let cbor = fvm_ipld_encoding::to_vec(&in_memory).unwrap();
assert_eq!(
    cbor,
    0b_11011000_00101010_01000101_00000000_00000001_00000000_00000000_00000000_u64.to_be_bytes(),
);

In rust, the most common serialization framework is serde. It has ONE (de)serialization model for each struct - the serialization code cannot know if it’s writing JSON or CBOR.

The cleanest way handle the distinction would be a serde-compatible trait:

pub trait LotusSerialize {
    fn serialize_cbor<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
    where
        S: Serializer;

    fn serialize_json<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
    where
        S: Serializer;
}
pub trait LotusDeserialize<'de> { /* ... */ }

However, that would require writing and maintaining a custom derive macro - can we lean on serde::Serialize and serde::Deserialize instead?

§Lotus JSON in Forest

Whenever you need the lotus JSON of an object, use the LotusJson wrapper. Note that the actual HasLotusJson::LotusJson types should be private - we don’t want these names proliferating over the codebase.

§Implementation notes

§Illegal states are unrepresentable

Consider Address - it is represented as a simple string in JSON, so there are two possible definitions of AddressLotusJson:

#[derive(Serialize, Deserialize)]
pub struct AddressLotusJson(#[serde(with = "stringify")] Address);
#[derive(Serialize, Deserialize)]
pub struct AddressLotusJson(String);

However, with the second implementation, impl From<AddressLotusJson> for Address would involve unwrapping a call to std::primitive::str::parse, which is unacceptable - malformed JSON could cause a crash!

§Location

Prefer implementing in this module, as decl_and_test will handle quickcheck-ing and snapshot testing.

If you require access to private fields, consider:

§Compound structs

§Optional fields

It’s not clear if optional fields should be serialized as null or not. See e.g LotusJson<Receipt>.

For now, fields are recommended to have the following annotations:

ⓘ
#[serde(skip_serializing_if = "LotusJson::is_none", default)]
foo: LotusJson<Option<usize>>,

§API hazards

  • Avoid using #[serde(with = ...)] except for leaf types
  • There is a hazard if the same type can be de/serialized in multiple ways.

§Future work

  • use proptest to test the parser pipeline
  • use a derive macro for simple compound structs

Modules§

Macros§

Structs§

  • LotusJson
    A domain struct that is (de) serialized through its lotus JSON representation.

Traits§

Functions§