Monday, February 5th, 2018

Improving the web with small, composable tools

Firefox Screenshots is the first Test Pilot experiment to graduate into Firefox, and it’s been surprisingly successful. You won’t see many people talking about it: it does what you expect, and it doesn’t cover new ground. Mozilla should do more of this.

Small, Composable Tools

One of the inspirations for Firefox Screenshots was user research done in 2015. This research involved interviews with a few dozen people about how they save, share, and recall information. I myself had a chance to be part of several house visits in Rochester, NY. We looked over people’s shoulders while they showed us how they worked.

My biggest takeaways from that research:

  • There is a wide variety of how people manage their information, with many combinations of different tools and complex workflows
  • Everyone is pretty happy with what they are doing
  • People only want small, incremental changes
  • Screenshots are pretty popular

It was surprising to see how complicated and sometimes clearly suboptimal people’s workflows were, while also understanding that each person was happy with what they did. They were happy because they weren’t looking for something new. At any moment most people are settled (satisficed) on a process, and they have better things to do than constantly reconsider those choices.

After learning how they worked, we’d sometimes offer up alternatives and get reactions. The alternatives received lots of crickets. If you could add a tool to existing workflows then there might be interest, but there wasn’t interest in replacing tools unless perhaps it was a one-to-one match. People specifically weren’t interested in integrated tools, ones that improved the entire workflow.

And who among us hasn’t been burned by overenthusiasm for a fully integrated tool? It seems great, then it gets tiring just to keep track, annoying to try to get people to sign up so you can collaborate, some number of things don’t fit into the process, you’ve lost track of your old things, it just feels like work.

Old Philosophies

The Unix philosophy:

  • Write programs that do one thing and do it well.
  • Write programs to work together.
  • Write programs to handle text streams, because that is a universal interface.

This is still what works well, and still what people want! This is also what the web can provide and apps and silos cannot: open composability.

This isn’t the same as APIs and integrated tools: find and grep are not integrated, you don’t have to setup OAuth integration between tail and tee. Things work together because you use them together.

What would the Unix toolset look like on the web? Please speculate! I’ve started structuring some of my own ideas into a set of notes.

Stop Being So Clever

At the time of the user research myself and Donovan had been working on an experiment in page capture – you could think of it like a personal archive.org. We added screenshotting as an entree into what felt like a more advanced tool.

In the end nothing is left of that original concept, and we just have plain screenshots. It hurt to see that all go. Screenshots are not exciting, and they are not innovative, and there is nothing very new about them. And clearly I needed to get over myself.

And so this is a lesson in humility: things don’t have to be new or novel or exciting to be useful. Screenshots is so un-new, so un-novel, so un-exciting that we aren’t even following along with the competition. Mozilla should spend more time here: behind the curve where the big players stopped caring and the little players have a hard time getting any attention. Behind the curve is where the web was a lot more like how Mozilla wants it to be.

There are lots of useful things back here, things that technophiles have appreciated but the wider population doesn’t know how to use. A pastebin. Site archival. Deep linking. Inline linking at all! Scraping. Clipboard management. Etherpad is still the best lightweight collaborative editor. Little stuff, things that don’t try to take over, things that don’t try to leverage the user for corporate benefit. This stuff is not very hard to make, and is affordable to run. Combine that with a commitment to keep the services competently maintained and openly interoperable, and there’s a lot of value to provide. And that’s what Mozilla is in it for: to be of service.

Being Part Of The Web

Screenshots was not easy to make. It was not technically difficult, but it was not easy.

Mozilla has long been reluctant to host user content. Firefox Sync is pointedly encrypted on the client. Before Screenshots the only unencrypted user content the corporation handled was the add-ons and themes on addons.mozilla.org.

Screenshots did not have to have a server component, and it did not have to allow people to upload or share shots within the tool. I take some pride in the fact that, despite all our cultural and legal attitudes at Mozilla, screenshots.firefox.com is a thing. It required a great deal of stubbornness on my part, and at times a pointed blindness to feedback.

In a small way Screenshots makes Mozilla part of the web, not just a window onto the web. This is a direction I think we should take: *.firefox.com links of all kinds should become normal, and you should know that on the other side of the link will be respectful content, it won’t be an avenue for manipulation, and you won’t be a product. Be the change you want to see, right?

Thanks to Wil Clouser and Jared Hirsch for feedback on this post.

Comments

ShalabhThu, 08 Feb 2018

I really love Firefox screenshots and is one reason I've switched to Firefox almost completely! Other useful tools would be some kind of annotation system. I use hypothes.is and it would be great if it was better integrated.

You asked what a Unix toolset would look like on the web. This is hard because many things may not end up being incremental, but if we had to start, I imagine something like the following (btw, I'm not a fan of text streams, but it doesn't affect the bigger idea here):

- You have URLs that point to 'content' and URLs that point to 'tranformers'.

- You transform content by piping it through the tranformer. For instance, I type this in my browser: `https://my/data | https://other/transform?param=value` - it converts the data via the transformer and displays it in the browser.

- To save the data, you pipe to another special URL, e.g. `https://my/data | https://other/transform | https://my/data2`

- A possible mechanism is GET my/data, POST other/transform and then POST my/data2.

- Auth is handled transparently.

- 'Built-in' transformers could be provided by special `about:` style URLs.

- Each URL restricts the schema/format of data you can PUT under it. E.g. I could implement http://my/books/ such that it only accepts specific JSON. This allows a range of services from 'blob store' to
'custom database' to be implemented within this scheme. Extending this idea to 'content streams' - a URL could return a stream of content and the transform would apply to each item, you could filter it, etc.

- Features like 'search' then become just a URL that return a stream of content.

This does require a wide array of services to provide and consume data in a specific protocol. But if it catches on, and comes with a nicer UI than the JS console, it can becomes quite useful.

Ian BickingMon, 12 Feb 2018

Lots of interesting ideas here!

One thought I have, looking at your ideas: can all of these be reified as web pages? If you make something into a page it makes it clearer what's happening, and encourages mixing and matching of tools. For instance: what does pipe look like as a web page?

Another thought: what does abstract vs concrete composition look like? By concrete, I mean performing actions. E.g., go to https://my/data, copy the URL, open https://other/transform, paste the URL into a form, submit. That's the normal way we compose services, and I think we'd want that – normal also means expected, and transparent, and discoverable. Is there a natural bridge from the manual approach to a more structured or automated or describable approach?

Can we consume the existing streams we have? E.g., take the search services we have available right now, get smart enough to tell what the result links are in the page, and smart enough to understand pagination, and then... we're good? Other services aren't allowed to just scrape Google search results, but other services aren't user agents, we're applying understanding purely (we'd hope!) for the benefit of the user, who after all was always allowed to see the search results.

Combine reification and operating on existing services, and you get something a little like http://lmgtfy.com/ – not fancy, but just a simple way of sharing not just results, but how the results were attained.

> This does require a wide array of services to provide and consume data in a specific protocol. But if it catches on, and comes with a nicer UI than the JS console, it can becomes quite useful.

Do we need new services and protocols? I suspect, if we're assertive enough, that we can discover protocols in the services we already have. But probably we'll need just a little more glue to make it all come together.

ShalabhThu, 15 Feb 2018

> can all of these be reified as web pages?

Hmm, perhaps. Things like auth and cross origin policies typically get in the way.

> Is there a natural bridge from the manual approach to a more structured or automated or describable approach?

Ah interesting question. Thinking about this, it starts to look like 'macro recording' in some editors and environments. You 'start recording', do your thing, 'stop recording', then view the workflow that you recorded. You then can just rerun or parameterize and rerun with different inputs.

> Do we need new services and protocols?

Not strictly, but the issue with the web as-is is discover-ability. For instance how can the browser know which fields an HTTP POST endpoint accepts? If it's a form it may be able to extract that, but what if the data is constructed via javascript? I wasn't thinking of a completely new protocol but a scheme on the existing protocols. For instance a well defined way to authenticate and to determine what format/schema of data can be posted to a URL. This might help if the browser can build a registry of 'urls that accept pictures', and then whenever it has an picture object, allow piping to any of those.

What I meant with new services is if we have a large number of third party providers running services using a scheme that is easily integrated into the workflow, with auth and piping stuff around, then the value of the ecosystem becomes much larger. I'm not aware of any services presently. Most are designed for manual operation via custom logins and very specialized web apps.

lebauxTue, 13 Feb 2018

While I like the idea that firefox is becoming more of a complete tool, I prefer modularity. That’s why addons exists, no? At its current form, screenshots are redundant component, as it simply isn’t doing enough. I need to annotate, therefore use another extension or snag.gy. While I don’t hate native firefox screenshots at its current form, I don’t believe adding set if simple annotation options would hurt workflow of anyone, as it would be something you could do after the screen is already uploaded and it would be completely optional extra step. So yeah, “do one thing and do it well” simply ins’t the case for screenshots yet and this reads like you are assuring yourself that the tool in its current form is what majority wants, not you. Snag.gy does one thing, “reasonably” well and it has been around for ages. The whole thing is a website, all I need is a bookmark. I think it is way more elegant, like tools you mentioned — Etherpad.

It goes with Send, too. Both seems to focus on privacy with no accounts and expiration dates. I imagine this aligns with Mozilla’s way, but it is not terribly convenient. At the same time, garbage like Pocket, Reader+, WebRTC (arguably useful) and Hello went right into the firefox. And you already store my bookmarks, passwords (well not mine, but lot of people opt-in here) via sync, so why not just enrich the experience here. People allow amazon to go inside their house for deliveries, disclose personal and sensitive info everywhere and don’t care about privacy in general. Why can’t I login with my sync account and have granular control? Or the problem is people would take up storage with their screenshots?

I don’t know, it just feels like Mozilla’s vision/products are split in a way and I already feel like Vivaldi is doing better job on the privacy front. As a user I feel more in control of what components of the ecosystem I use. But that is just me.

Ian BickingTue, 13 Feb 2018

> While I like the idea that firefox is becoming more of a complete tool, I prefer modularity. That’s why addons exists, no?

Screenshots is an add-on, but unfortunately for boring reasons we couldn't expose it as an installed add-on that you can disable. At least not yet.

> I need to annotate, therefore use another extension

BTW, we'll be adding this feature to Screenshots in the next couple months.

I would have liked if annotation was a separate feature, complementary to but separate from Screenshots. But there's a lot we'd have to figure out to do that, and so we've moved forward in a conventional way, making it a feature built on a feature.

Similarly, I can see how snag.gy separates the upload/publish step from the screen capture in an interesting way, where Screenshots combines the two. I like that... but I think we'd need to figure out the metaphors and flows some, and in the interest of moving forward we used more conventional approaches.

I don't want to suggest that the tools I am imagining don't already exist, or that they each need to be integrated directly into Firefox. I think there's an opportunity to introduce those tools to more people, and complement them with a few tools that don't exist or are quite obscure. (For instance the browser itself can be very non-webby, which is one issue I see.)

> It goes with Send, too. Both seems to focus on privacy with no accounts and expiration dates

Expiration dates were put in when Screenshots was an experiment, because... we didn't want to commit to anything long term. Then no one complained, and many people even seemed to appreciate the expiration. I was a bit surprised. The expiration can be extended.

> Why can’t I login with my sync account and have granular control?

That feature will be coming too.

Requiring sign-up to try something out is a pattern we wanted to avoid, so we started with a semi-anonymous mode (there is a sort of Screenshots account tied to your browser profile). People use accounts for lots of different purposes, so we've had a hard time coming up with a unified approach to accounts in Firefox. People are familiar with web logins, but a browser login doesn't quite fit that model, and so far we've reverted to using web logins for additional services even when you are logged into Firefox (which is also how Screenshots will work, at least to start). I'm not happy with it, but it's a morass I don't want to get caught up in right now.

tweets

This is the personal site of Ian Bicking. The opinions expressed here are my own.