July 24, 2015

Recipe Architecture

  1. Backend
    1. Backend Handler
    2. Backend Entities
    3. Backend Validation
    4. Backend Storage
  2. Frontend
    1. Frontend Dispatcher
    2. Frontend Entities
    3. Frontend Validation
    4. Frontend Storage
  3. Things to think about

Since my last post, I've been working a bit on the architecture of the recipe app.

Backend

Backend Handler

I need some handler for a websocket/longpolling RPC interface. I'll use sente, so I'll need a handler for events. Basically a dispatcher, it should figure out what to run based on the id of the event. For example the :account/create event should create a account, with data maybe looking like {:email "[email protected]" :password "test"}.

Backend Entities

These are entity-specific handlers. :account/create will validate the account, and if it's valid, persist it to the database.

Backend Validation

Validators hold business logic, for validating new/updated entities. The websocket handler will dispatch something like [:account/create {:email "[email protected]" :password "test"}] to the account namespace. They answer questions like:

  • I have a account. Is it valid? For example, is its email address a valid length? Does it even have an email address?
  • Is my account's email a duplicate of another account's? If so, it's invalid.
  • Is this a valid length for an email address?
  • I have a password and a account. Is the password correct?
  • Is this recipe missing any required fields, like a title?

Backend Storage

This gives us access to the database--inserts, updates, and selects of valid entities all go through here.

Frontend

Let's look at re-frame, a reference implementation of a reactive-UI programming model. re-frame is to reagent what Flux is to React.

re-frame's README is long for a reason. It's well worth reading. There's very little magic involved in re-frame—even though this description will seem magical.

The short, oversimplified, inaccurate version of what re-frame is about is:

  • components take the current application state (the database, or "subscriptions" into it, but we'll ignore that) and return DOM, and react to database changes by rerendering new DOM.
  • handlers are functions of the current application state and arbitrary additional args, and return a new application state

So think of it like this going one way (from state to HTML):

application state ->           component                -> HTML
{:name "Bob"}     -> (fn [] [:div "Hi, " (:name @db)])  -> <div>Hi, Bob</div>

... and like this the other (from events to state):

event                              ->           (handler state some-data)          -> new state
(dispatch [:name-changed "Harry"]) -> (fn [db new-name] (assoc db :name new-name)) -> {:name "Harry"}

... after which the state -> DOM picture looks like:

application state   ->           component                -> HTML
{:name "Harry"}     -> (fn [] [:div "Hi, " (:name @db)])  -> <div>Hi, Harry</div>

There's a lot more to it than that, and again, README! But that's the 10-second version.

Now, there's some interesting parallels here--let's walk through the similarities and differences between the front and back-end code, so we know what we can reuse.

Frontend Dispatcher

First of all, (dispatch [:name-changed]) (or perhaps (dispatch [:account/create {:email "[email protected]" :password "test"}])!) looks suspiciously similar to our websocket handler on the backend. But careful! Some events dispatched on the client can't be handled by the server. For example, we might implement undo on the client by recording the current state and reverting to it if necessary. On the server, we can't do that (even if I were using Datomic and could revert the database state, doing so might result in a bad experience for all the other users besides the one who clicked "undo"!).

So, sending the server our dispatched events directly is out. What we can do is watch the client's state ratom (or subscriptions to it?) for changes, construct events from those changes, and send them to the server. For example, say the client reorders a list of recipes: {:recipes [1 2 3]} becomes {:recipes [1 3 2]}

    user=> (diff {:recipes [1 2 3]} {:recipes [1 3 2]})
    ({:recipes [nil 2 3]} {:recipes [nil 3 2]} {:recipes [1]})

We can use these changes to construct events to send to the server, something like

    [[:recipe/reorder 2 {:from 1 :to 2}]
     [:recipe/reorder 3 {:from 2 :to 1}]]

(which captures the fact that 2 moved from index 1 to 2, and 3 switched from index 2 to 1.)

In reality, because of the mechanics of reordering, recipe ordering should be probably done with a linked list to minimize the number of changes necessary for each diff, but this example works for now.

Frontend Entities

Assuming we can write a function to turn a state change into a neat dispatchable event like [:account/create {:account :data}], the handler logic should be able to stay the same: if the change is valid, persist it.

The only difference is that if we're on the server, and we have a ?reply-fn, we'll call it with the results.

One thing to note though is that on the server, we'll want to reply with data on success or failure: {:account-created {:account :data}} or {:error {:some :error}}. On the client, we return a new application state--if there's an error, or if the creation succeeded, that will be reflected in the application state. That seems to result in error notification happening in storage on the client, and the entity on the server--I'll have to think more about this.

Frontend Validation

Many, but not all, of our validation questions can be asked from the front-end as well: we don't want to accept an account with a blank email address on either end, for example. On the other hand, unless we plan to load every account into memory on the client, we can't exactly check for duplicate email addresses on the client side. And we definitely don't want to check passwords by comparing the hashes on the client!

Frontend Storage

Storage will also look different on the client and the server: the client "database" is a big ratom, while the server database is, well, a database. To get data from the database on the client, we use reactions, a stream of values that we just dip into whenever we want to get the current state. On the server, we have to explicitly request data, and send it to the client.

And modifying the database is quite different: on the client, we simply define functions of one state that return a new state. On the server, this must be done with side effects.

Things to think about

  • What happens if the server/client go out of sync? E.g. if client fails to handle an error response from the server correctly. How do we notice this? How do we fix it? How do we prevent it?
  • What's the best way to go about converting a diff to a clean event?
  • Does order matter for applying changes? (Yes.) Can we ensure that the server processes changes in the same order they occurred on the client?
Tags: recipe-app