Serialization and reconstruction: rebuild objects from saved data

Rebuild library objects from saved plain data through a validating factory, and see what a rebuild loses: identity, runtime state and sharing. Methods and type contracts · Object-oriented programming.

A librarian saves the library as data and loads it back the next morning. Mira's loan says Dune was returned, but the reloaded catalog still lists Dune as out. Nothing crashed, and every title and number came back. What did loading rebuild, and what did it leave behind?

Saved data describes state, nothing more

To keep a library between runs, we write it down as plain data: dictionaries, lists, text, numbers and true or false. Plain data describes state. It holds no object, no class and no method. Loading reads that description and builds new objects from it. This lesson is Python-only.

The Library from earlier lessons now supports saving.

LIMIT = 2 is the borrowing limit.

LIMIT = 2

class Book:
    def __init__(self, title, available=True):
        self.title, self._available = title, available
    def is_available(self): return self._available
    def mark_in(self): self._available = True
    def to_data(self, bid):
        return {"id": bid, "title": self.title, "available": self._available}

class Loan:
    def __init__(self, book, due_in_days): self.book, self.due_in_days = book, due_in_days

class Member:
    def __init__(self, name, loans=()): self.name, self.loans = name, list(loans)
    def active_loans(self): return len(self.loans)

Book reads its own private _available inside to_data, so the saver never reaches into a private field.

class Library:
    def __init__(self, books, members): self.books, self.members = books, members
    def to_data(self):
        ids = {b: f"b{i + 1}" for i, b in enumerate(self.books)}
        def loans(m):                      # one member's loans, as plain dictionaries
            return [{"book": ids[l.book], "due_in_days": l.due_in_days} for l in m.loans]
        return {"books": [b.to_data(ids[b]) for b in self.books],
                "members": [{"name": m.name, "loans": loans(m)} for m in self.members]}

dune, atlas = Book("Dune", available=False), Book("Atlas")
live = Library([dune, atlas], [Member("Mira", [Loan(dune, 7)])])
doc = live.to_data()

Each book gets an id such as "b1", counting up from the first book, and a loan names its book by that id. As JSON, doc reads:

{
 "books": [
  {
   "id": "b1",
   "title": "Dune",
   "available": false
  },
  {
   "id": "b2",
   "title": "Atlas",
   "available": true
  }
 ],
 "members": [
  {
   "name": "Mira",
   "loans": [
    {
     "book": "b1",
     "due_in_days": 7
    }
   ]
  }
 ]
}

Name the rules before the code

A file can hold anything, so loading must not trust it. The schema is the agreed shape: which keys each record has and what type each value is. We state it and six rules in a fixed order. The first rule a record breaks is the one reported.

R2 checks a whole number with type(n) is int, not isinstance(n, int). In Python, bool is a kind of int (a subclass), so True counts as the number 1. A run printed isinstance(True, int) as True and type(True) is int as False, so the weaker check would accept True as a loan length.

Check first, then build

A factory is a function that builds objects for us. Ours is a validating factory: it checks the data before it builds anything. Loading has two phases. Phase one is validate: it runs every rule over the whole record, raises ValueError at the first broken one, and builds nothing. Phase two builds, only for a record that passed.

def validate(d):
    def rule(ok, name):
        if not ok: raise ValueError(name)
    rule(set(d) == {"books", "members"}, "R1 shape")
    bs, ms = d["books"], d["members"]
    rule(all(set(b) == {"id", "title", "available"} for b in bs)
         and all(set(m) == {"name", "loans"} for m in ms), "R1 shape")
    ls = [l for m in ms for l in m["loans"]]
    rule(all(set(l) == {"book", "due_in_days"} for l in ls), "R1 shape")
    rule(all(isinstance(b["id"], str) and isinstance(b["title"], str)
             and type(b["available"]) is bool for b in bs)
         and all(isinstance(m["name"], str) for m in ms)
         and all(isinstance(l["book"], str) and type(l["due_in_days"]) is int
                 for l in ls), "R2 types")
    by_id = {b["id"]: b for b in bs}
    rule(len(by_id) == len(bs), "R3 ids")
    rule(all(len(m["loans"]) <= LIMIT for m in ms), "R4 limit")
    rule(all(l["book"] in by_id for l in ls), "R5 reference")
    rule(all(1 <= l["due_in_days"] <= 14 and not by_id[l["book"]]["available"]
             for l in ls), "R6 meaning")

The sketch assumes parsed dictionaries and lists. rule(ok, name) is a small helper that raises ValueError(name) when its check is false. R1 is split in three so that a missing key is reported as R1, not as a KeyError.

Why check everything before building? A loader that builds as it reads can stop at record three with two objects made and no clear state. A loader that checks first builds everything or nothing.

Watch the rules pass, then the loan look its book up by id.

def library_from_data(d):
    validate(d)                                  # phase 1: check everything, build nothing
    table = {b["id"]: Book(b["title"], b["available"]) for b in d["books"]}
    people = [Member(m["name"], [Loan(table[l["book"]], l["due_in_days"]) for l in m["loans"]])
              for m in d["members"]]
    return Library(list(table.values()), people)

new = library_from_data(doc)
print(new.members[0].loans[0].book is new.books[0])   # one Dune, held by loan and catalog
print(new.books[0] is dune, new.books[0].title == dune.title)

# a copy of doc whose only change is a loan naming a book "b9" that does not exist
bad = {**doc, "members": [{"name": "Mira", "loans": [{"book": "b9", "due_in_days": 7}]}]}
try:
    library_from_data(bad)
except ValueError as e:
    print("refused:", e)

The run printed True, then False True, then refused: R5 reference. The loan and the catalog share one Dune, which is not the original.

What comes back is new

Equal state is not the same object, as identity and aliasing taught. The rebuilt Dune has the same title and availability, yet new.books[0] is dune is False. A load makes fresh objects with their own identities.

Only what the data names comes back. In a second run, a _scan_log attribute set on the original Dune was missing from the rebuilt Dune. The class and its methods come from your code, and a file or connection must be opened again by the factory.

The lost alias

Now save the library as a tree. Put Dune inside Mira's loan as well as in the catalog, and give books no ids. JSON has no way to say that two places are one object, so the loader builds the nested Dune as a second Book.

def library_from_tree(d):                        # no ids: each loan carries its own book
    def make(b): return Book(b["title"], b["available"])
    people = [Member(m["name"], [Loan(make(l["book"]), l["due_in_days"]) for l in m["loans"]])
              for m in d["members"]]
    return Library([make(b) for b in d["books"]], people)

tree = {"books": [{"title": "Dune", "available": False}, {"title": "Atlas", "available": True}],
        "members": [{"name": "Mira", "loans": [
            {"book": {"title": "Dune", "available": False}, "due_in_days": 7}]}]}
t = library_from_tree(tree)
t.members[0].loans[0].book.mark_in()             # Mira returns Dune through her loan
print(t.books[0].is_available())                 # the catalog still says out

The run printed False: the catalog's Dune is still out. That is the opening bug. The two Dunes look equal but are two objects, so a change through one is invisible through the other.

A tree loader needs its own rules: R3 and R5 have no subject without ids, and no rule compares the two Dunes.

With ids, the same return worked: the id-based rebuild reported the catalog's Dune as available after the loan returned it. Whether an alias survives is a choice of the format, the alias policy. A tree drops it. Ids keep it, because "b1" appears in two places and the factory resolves both to one object. A deep copy in a running program, from the copying lesson, keeps sharing by remembering what it copied. A tree dump has no such memory.

A refused record builds nothing

Back to ids, with a loan that names b9. R1 to R4 pass, R5 fails, and the factory raises before it builds.

In a run that counted calls to Book.__init__, the refused load made 0 Books and the accepted load made 2. The live library was untouched: live.to_data() == doc stayed True. Raising ValueError is the factory's contract, as in exception contracts. The caller keeps its valid library.

Why not pickle?

Python's pickle saves objects themselves and keeps sharing within one dump. The catch is that ordinary unpickling does not call __init__, so no validation runs.

import pickle

class PBook:
    def __init__(self, title): self.title = title

class PMember:
    def __init__(self, loans):
        if len(loans) > 2: raise ValueError("limit is 2")
        self.loans = loans

dune = PBook("Dune")
mira = PMember([dune])
mira.loans += [dune, dune]                     # a state the constructor would refuse
back = pickle.loads(pickle.dumps(mira))        # no error: __init__ is not called
print(len(back.loans), back.loans[0] is back.loans[1])

The run printed 3 True. The member with three loans loaded without any check, and the sharing came back. The same state through library_from_data is refused by R4. Never unpickle data you do not trust, because loading can run code chosen by whoever wrote it. For data from outside your program, use plain data and a validating factory.

Recap

• Loading builds new objects with equal state and new identity, and nothing the data did not name.
• A factory checks the whole record first, so a refused record builds nothing.
• Sharing survives only where the format names it. A tree repeats Dune and rebuilds two objects, which is the stale catalog. Ids resolve to one.
• For outside data, use plain data and a validating factory. pickle runs no checks and must not load untrusted data.

This closes the optional module on methods and type contracts. To practise telling one shared object from two equal copies, try Linked List Fundamentals, where one node can be reached from two places.