Note
Translated from the French by Dan Phrawzty.
Mozilla has a tradition of "work weeks" whereby the members of our many distributed teams get together in a single physical location a few times per year; occasionally these work weeks involve the entire company, as was the case most recently in Whistler, BC, Canada.
It was a great opportunity for our team to come together, to share our vision and ideas concerning data storage, and to collect use-cases for Kinto.
Workshops and Self-Promotion
Nicolas presented Kinto.js at a dedicated workshop in Whistler, the content of which now forms a introductory tutorial. The resulting application, simple though it appears, is a great way to learn the basic concepts behind synchronisation in Kinto. Furthermore, we didn't even need to set up a server - RĂ©my set one up that resets itself every day!
As an aside, we chose to use Vanilla-JS for two reasons: firstly because it allowed us to avoid the framework wars, but also because it proves that we can do so much more with HTML5 and ES6 than was possible a few years ago.
Furthermore, this small workshop helped us to realise that we still have large gaps in our documentation, particularly as regards the eco-system and the long-term, global vision for our family of projects. This is an area where we have resolved to do better.
Mozilla Payments
As noted in a previous blog post we developed a permissions model which allows Kinto fulfill the the requirements of payment and subscription tracker (our fist use-case).
For this project, Kinto is going to be used by a Django app via a Python client.
Now that the development is done we have to deliver - that means integrating, hosting, and really pushing the usefulness of the stack. The solution is should be ready to go by the end of the year.
Coming Soon
Following the recent successful implementation of a permissions model, we'd like to take some time out to implement a feature that is important to us: allowing read access to shared records in a collection. If this is interesting to you, feel free to follow the issue or (better yet) contribute a patch!
Firefox OS and Storage
We've collaborated frequently with the Firefox OS team over the years, notably regarding the MISISDN Gateway (basically a way to link BrowserID and SMS together) and Firefox Hello.
In-App Sync
Kinto as a simple solution for synchronising data in Firefox OS? Nice! We've had this partnership in mind for some time (remember Daybed?), and now we've got a great opportunity at our fingertips.
We'll need to clearly and plainly lay out the limitations and simplistic models inherent in our solution (notably regarding concurrency management), but we believe that it fits the use-case fairly well, so it should not disapoint. :)
We're particularly motivated as both Dale Harvey (one of the authors of PouchDB) and Michel de Jong (one of the authors of Remote Storage) have given us plenty of encouragement from an early stage.
Cut The Rope
Kinto is likely to be used to synchronise the configuration settings and scores of the very popular game Cut The Rope. It'll be good to have a practical use-case on such a visible platform.
Syncto
Firefox Sync is what enables synchronisation between Firefox (favourites, extensions, browser history, etc.) and many other targets in an encrypted fashion. The current Javascript client implementation is relatively complex and, frankly, somewhat dated (the proposed re-write has yet to occur).
We'd like to implement a bridge between Kinto and Firefox Sync via Kinto.js - which is more simple and more modern than the existing reference client - in order to store synced data in IndexedDB. On the server side, we feel that this shouldn't be too hard since our project is itself inspired by Sync's proven protocol model; from a client perspective, most of the work will be in plumbing crypto into BrowserID authentication.
Alexis jumped at the chance to start work on a Python Firefox Sync client which will serve as the base for any future service.
Cloud Storage
Eden Chung and Sean Lee did a presentation on their advancements regarding the integration of remote storage services (notably DropBox and Baidu Yun) in Firefox OS. Their current proof-of-concept is based on FUSE.
We were also thinking about introducing the idea of attachments in Kinto by implementing the remote storage specification; however, for the moment our existing use-cases just aren't calling for it.
Coming Soon
In addition to the work that we've already done on the server side, we're going to have to deal with concurrency management in our Javascript client in order to allow simultaneous writes and background synchronisation.
We are always looking for feedback - and, of course, code contributions - on both the server and client!
Firefox Application Content
The Firefox development and release process is currently based on a six-week cycle. One of the objectives is to decouple certain specific content elements (such as safety rules, dictionaries and translations, etc.) from these relatively long cycles [1].
The proposed solution is versioned, read-only JSON and binary blobs that can be synchronised with live browsers. There are already a number of tools to manage this sort of thing (examples include Balrog and Shavar), but for the moment no choice has been made. During conversations with the team in charge of this project it became evident that Kinto could be useful here too - which is motivating us even more to evolve our project!
| [1] | The good news is that all the existing 3rd-party functionality will be revived in the form of add-ons. |
Awesome Bar
The Firefox Labs team, best known for raising red pandas in test tubes, is interested in our solution as well, notably as regards the evolution of the Awesome Bar which would meld URLs, browser history, and search functions.
We can't say too much right now, but the aforementioned shared collections feature in Kinto would fit so very nicely into the future of the Firefox browser. :)
Coming Soon
In all likelihood we will need to implement indexing and full-text searching (read: Elasticsearch) before the end of the year. This fits nicely into our roadmap since it's something that we had in Daybed already.
Browser.html
The Search team has been exploring and extending the very notion of a platform, and is currently working on implementing a web browser in Javascript and HTML5 with React, the result of which is affectionatly known as browser.html.
Kinto aligns very well with the needs of that project as regards the synchronisation of user data. This could be as simple as a replication of Sync itself, but it could also be something more interesting, such as entire collections of any arbitrary data - for example, browser preferences, or an Alexa-top-500-style mechanism that would allow URL completion without the need to send a search request.
But why stop with just the local browser? We could synchronise entire React states between peripherals, thus allowing a seamless browser experience across every device!
Coming Soon
If Browser.html must store navigation data, we believe that it should do so using strong encryption directly in the Javascript client. Luckily we're not the only ones thinking about this, and there are already several possible standards.
To avoid pinging the server at regular intervals in order to synchronise the changes (effectively DDoS'ing ourselves!), the introduction of push notifications seems like a natural choice. This would be the final building block in our quest to build a complete "Mobile / Web backed as a service".
Conclusion
We're in pretty much the ideal situation right now: everything that we've imagined, worked on, prototyped, and shipped corresponds to the needs and desires of a number of teams at Mozilla.
Our challenges ahead are:
- Co-ordinate with the other teams efficiently.
- Avoid disappointment.
- Maintain a high level of productivity.
- Continue to improve and promote our solutions.
- Focus on quick wins that move us forward.
Finally, we will work to encourage meaningful community contributions to help us build a free, generic, simple, and self-hostable solution for storing data on the web.
