How to load test a OAuth-protected HTTP API? (Part 2)

This is part 2 of a series on loadtesting OAuth APIs.

Introduction

In the previous article we have seen that:

We should create the OAuth Bearer Token before the load test start.
We can use environment variables to specify the token to use in the load test.

In this article we will have a closer look at how to generate tokens for Firefox Accounts and pass them to the load test.

PyFxA: The Python Firefox Accounts client

PyFxA is a Python library providing helpers to interact with the Firefox Accounts ecosystem.

Using PyFxA with httpie

To ease the use of Firefox Accounts Bearer Tokens, we have created, a while ago, an authentication policy for the requests library, which can also be used with the httpie client.

With this plugin, we can provide our user credentials and a valid Bearer Token will be built for our request.

Here is what it looks like with HTTPie.

http https://profile.accounts.firefox.com/v1/profile \
    --auth-type fxa-bearer \
    --auth "email@domain.tld:password"

Using fxa-client to generate Bearer Tokens

The code we wrote at the time was quite similar to what we were trying to do for the load test setup: generating a Bearer Token for a given user.

We ended-up building a CLI tool called fxa-client that is able to generate a BASH file that exports some environment variables:

USAGE: fxa-client [-h] [--bearer] [--create-user] [--auth AUTH]
                  [--out OUTPUT_FILE] [--verbose] [--user-salt FXA_USER_SALT]
                  [--env {stable,stage,production}]
                  [--user-email-prefix PREFIX] [--account-server ACCOUNT_SERVER_URL]
                  [--oauth-server OAUTH_SERVER_URL] [--client-id CLIENT_ID]
                  [--scopes SCOPES]

With this client, instead of building a Bearer Token for the request, we can just ask the fxa-client to build one for our load test:

fxa-client --auth email@domain.tld --bearer --scopes "profile"

Please enter a password for email@domain.tld:
# ---- BEARER TOKEN INFO ----
# User: email@domain.tld
# Scopes: profile
# Account: https://api-accounts.stage.mozaws.net/v1
# Oauth: https://oauth.stage.mozaws.net/v1
# Client ID: 5882386c6d801776
# ---------------------------
export OAUTH_BEARER_TOKEN="b1d4babc01502fa8d9fc0139757168bf5908f33abc66f46253d4c69468e39373"

Then we can use this Bearer Token for our request:

http https://profile.stage.mozaws.net/v1/profile \
    Authorization:"Bearer b1d4babc01502fa8d9fc0139757168bf5908f33abc66f46253d4c69468e39373"

Using fxa-client to create new test accounts

We can also create a random user (on the stage environment only) and generate a Bearer Token for it:

fxa-client --create-user --bearer --user-email-prefix my-app

# ---- BEARER TOKEN INFO ----
# User: my-app-6318a65dde1efc2f4c3f7b4e6cb33188@restmail.net
# Scopes: profile
# Account: https://api-accounts.stage.mozaws.net/v1
# Oauth: https://oauth.stage.mozaws.net/v1
# Client ID: 5882386c6d801776
# ---------------------------
export OAUTH_BEARER_TOKEN="90abc87ed1621ee504c1252ed382abc8269d1abc29f2ff87cc5e25f00249fabc"

To avoid creating extraneous user accounts, it is possible to specify a user salt (as a base64 string) that will always generate the same user credentials and recreate the account if needed:

fxa-client --create-user --bearer --user-email-prefix my-app --user-salt MySalt==

Using fxa-client to work with BrowserID assertions

We are currently relying on OAuth2 Bearer Tokens for our new services.

However, some of the old services (Firefox Sync, Firefox Hello) still rely on BrowserID assertions for authentication.

Hopefully fxa-client is able to generate BrowserID assertions too.

In that case fxa-client provides few specific attributes:

optional arguments:
  --browserid, --bid    Generate a BrowserID assertion
  --audience AUDIENCE   Firefox BrowserID assertion audience.
  --duration DURATION   Firefox BrowserID assertion duration.

The script works exactly in the same way than the one for Bearer Tokens, except it generates a BrowserID assertion instead:

fxa-client --create-user --browserid --user-email-prefix my-app \
    --user-salt MySalt== \
    --audience https://loop.stage.mozaws.net

# ---- BROWSER ID ASSERTION INFO ----
# User: my-app-b82d4afaf57cb856ccc04a58a07ce80f@restmail.net
# Audience: https://loop.stage.mozaws.net
# Account: https://api-accounts.stage.mozaws.net/v1
# ------------------------------------
export FXA_BROWSERID_ASSERTION="eyJhbGciOiJSUzI1NiJ9...hIQ9vrkqA"
export FXA_CLIENT_STATE="828aef3bc68ac0bde10f3d4b93303088"

And then the assertion can be used in the Authorization header of the request:

http POST https://loop.stage.mozaws.net/v0/registration \
    Authorization:"BrowserID eyJhbGciOiJSUzI1NiJ9...hIQ9vrkqA"

Using fxa-client to configure a loadtest

Now that we have a quite simple way to generate Bearer Tokens, how can we plug that with our load tests?

I could not find a better way than creating a bash file that exports the environment variables and then sourcing it in the shell that will run the loadtest.

Something like:

fxa-client --create-user --bearer --user-salt MySalt== --out loadtest-fxa-config.sh
source loadtest-fxa-config.sh
docker run -e OAUTH_BEARER_TOKEN="${OAUTH_BEARER_TOKEN}" loadtest

After sourcing the loadtest-fxa-config.sh file, the env variables are exposed. Any program (regardless of the language) can read them if needed.

How to install fxa-client?

fxa-client have been released in the last release of PyFxA 0.1.0.

So just install the PyFxA python package using pip:

pip install -U PyFxA

Management Firefox Accounts environments

If we want to generate tokens for other Firefox Accounts environment, we would need to provide --account-server and --oauth-server which default to the stage environment.

We can find all the available environment here.

However to ease the switch from one to the other we added a --env parameter that let you write:

fxa-client --bearer --auth email@domain.tld --env production

Rather than:

fxa-client --bearer --auth email@domain.tld \
    --account-server https://api.accounts.firefox.com/v1 \
    --oauth-server https://oauth.accounts.firefox.com/v1

What's next?

Multiple accounts loadtest

We can already run the script twice to generate a Bearer Token per user, but it would be nice to be able to do so directly with fxa-client.

I was thinking of implementing the following output:

fxa-client --create-user --bearer --user-salt MySalt== -n 2

# ---- BEARER TOKEN INFO ----
# User1: my-app-1318a65dde1efc2f4c3f7b4e6cb33188@restmail.net
# User2: my-app-2318a65dde1efc2f4c3f7b4e6cb33188@restmail.net
# Scopes: profile
# Account: https://api-accounts.stage.mozaws.net/v1
# Oauth: https://oauth.stage.mozaws.net/v1
# Client ID: 5882386c6d801776
# ---------------------------
export OAUTH_BEARER_TOKEN="90abc87ed1621ee504c1252ed382abc8269d1abc29f2ff87cc5e25f00249fabc,abc9087ed1621ee504c1252ed382abc8269d1abc29f2ff87cc5e25f00249fabc"

Conclusion

That's about it. I hope that after reading this article, you are not afraid anymore of load testing Firefox Accounts OAuth-based services!

Take-aways:

fxa-client let us generate a bash script with our user credentials.
This bash script can be loaded before running our load test to expose user credentials to a load test script.

Do not hesitate to reach us if you have any questions or suggestions.

Revenir au début