Jan 2024

Streaming HTML out-of-order in Go

Check out the demo go-streaming-html-ooo.fly.dev to see the proof of concept. Source code for demo.

Inspired by Streaming HTML out of order without JavaScript article using Declarative Shadow DOM I wanted to try this out in Go.

Introduction

The basis of streaming HTML involves writing chunks of HTML. Here is a simple example in Go using go templates where we write the HTML in chunks.

func main() {
  http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
    t := template.Must(
        template.New("").Parse(`
      {{define "head"}}
        <html>
          <head>
            <title>Title</title>
          </head>
          <body>
      {{end}}
      {{define "content"}}
            Content
      {{end}}
      {{define "tail"}}
          </body>
        </html>
      {{end}}`)
      )

    t.ExecuteTemplate(w, "head", nil)

    time.Sleep(2 * time.Second)
    t.ExecuteTemplate(w, "content", nil)

    t.ExecuteTemplate(w, "tail", nil)
  })

  log.Fatal(http.ListenAndServe(":"+port, nil))
}

If you tried to open this in the browser, you would see the page only appears after 2 seconds. This is because the browser http.ResponseWriter does not flush the response buffer automatically unless the chunks are large enough. We can force the flush by explicitly calling Flush().

t.ExecuteTemplate(w, "head", nil)
w.(http.Flusher).Flush()

time.Sleep(2 * time.Second)
t.ExecuteTemplate(w, "content", nil)
w.(http.Flusher).Flush()

t.ExecuteTemplate(w, "tail", nil)
w.(http.Flusher).Flush()

Try the page again and you should see our content is successfully streamed to the browser. However for most applications this is not very useful. We want to stream the content out of order, as the content as it is available. In a real app, the app shell should load first with loaders, then the async content will stream in and be swapped.

Traditionally this is done with JavaScript that updates the DOM.

Declarative Shadow DOM (DSD)

Declarative Shadow DOM unlocks the ability to use the Shadow DOM on the server. You can do this with the shadowrootmode attribute on a <template> element.

Using this with Shadow DOM <slot> elements allows us to replace the contents of the <slot> element elsewhere in the HTML response from the server. This is the secret sauce that allows us to stream HTML out of order without JavaScript.

Let’s modify our previous template to use the Shadow DOM. I am also going to move the template into its own file for readability.

{{define "head"}}
<html>
  <head>
    <title>Title</title>
  </head>
  <body>
    <template shadowrootmode="open">
      <header>
        <h1>Header</h1>
      </header>
      <main>
        <slot name="content"> Loading content... </slot>
      </main>
      <footer>Footer</footer>
    </template>
    {{end}} {{define "content"}}
    <div slot="content">This will be rendered in the template slot</div>
    {{end}} {{define "tail"}}
  </body>
</html>
{{end}}

// Get template from the file
t := template.Must(template.New("index.html").ParseFiles("index.html"))

Now we can stream HTML out of order to the browser. Notice how the footer is rendered before the content.

Let’s try a more complex example where we stream a list of items out-of-order to the browser. I am not too familiar with Go so this might not be the best approach, but I use a channel for the async content. The data is sent to the channel in a delayed order.

// Async content
type Food struct {
  Id   int
  Name string
}

func sendFoodInDelayedOrder(data []Food, order []int, ch chan Food) {
  go func() {
    for _, index := range order {
      time.Sleep(500 * time.Millisecond)
      ch <- data[index]
    }
    close(ch)
  }()
}

var Foods = []Food{
  {Id: 1, Name: "Ice Cream"},
  {Id: 2, Name: "Pizza"},
  {Id: 3, Name: "Chocolate"},
  {Id: 4, Name: "Cheseburger"},
  {Id: 5, Name: "Oreo"},
}

We use this data in our template, with pre-created loader slots for each item based off the ids. The content is streamed to the browser in the order of the order array with some delay.

type AwaitedSlot struct {
  Slot string
  Html string
}

http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
  // ExecuteTemplate with head and content
  ...

  ch := make(chan Food)
  // Send the food to channel in a delayed fixed order
  go sendFoodInDelayedOrder(Foods, []int{4, 0, 3, 2, 1}, ch)
  // Stream the food to the browser, item by item
  for item := range ch {
    t.ExecuteTemplate(w, "slot", AwaitedSlot{
      Slot: fmt.Sprintf("slot-%d", item.Id),
      Html: item.Name,
    })
    w.(http.Flusher).Flush()
  }

  t.ExecuteTemplate(w, "tail", nil)
  w.(http.Flusher).Flush()
})

Template with slots for each item.

{{define "head"}}
...
{{end}}

{{define "content"}}
<div slot="content">
    <template shadowrootmode="open">
        <ul>
            <li>
                <slot name="slot-1">Loading...</slot>
            </li>
            ...
            <li>
                <slot name="slot-5">Loading...</slot>
            </li>
        </ul>
    </template>
{{end}}

{{define "slot"}}
<span slot="{{ .Slot }}">{{.Html}}</span>
{{end}}

{{define "tail"}}
<!-- Note that "content" slot is closed after item slots -->
</div>

</body>
</html>
{{end}}

The content loader “Loading content…” is displayed until the content is streamed in 1s later. Then the items are streamed in the order of the order array with 1s delay between each. The browser will swap the items at the appropriate list item loader.

Streaming HTML out-of-order in Go

Introduction

Declarative Shadow DOM (DSD)

References