In this blog post I want to showcase the integration system that I created for my self-tracking app using Golang and TypeScript. Seeing as I’m using my Fitbit smart watch for tracking exercise, sleep and general health, I figured it would be neat to pull this data automatically and avoid the need to type everything in manually.

An important property that I realized when planning this system was the extensibility aspect. It would be very tedious to have to write custom tailored implementations for every 3rd-party integration that I wanted to support. I instead dediced to opt for a more dynamic approach, defining integrations in a JSON structure.

{
    "entityType" : "sleep",
    "integrationType" : "FITBIT",
    "name" : "Sleep",
    "url" : "https://api.fitbit.com/1.2/user/-/sleep/date/#DATE#.json",
    "history" : {
        "url" : "https://api.fitbit.com/1.2/user/-/sleep/date/#START#/#DATE#.json"
    },
    "timestamp" : {
        "#date_time_notz" : "$.endTime"
    },
    "interval" : {
        "cron" : "0 */2 * * *",
        "jitter" : 0.0
    },
    "logSettings" : {
        "identifier" : "#DATE#"
    },
    "multiple" : "$.sleep",
    "identifier" : "$.logId"
    ...
}

As you can see, the 3rd party endpoint is entirely configurable and allows variables to fill in things like the current date. Here I can also define the interval at which I want to be fetching the data.

Extracting data from API responses #

Example API response from Fitbit:

{
  "dateOfSleep": "2025-07-25",
  "duration": 28620000,
  "efficiency": 95,
  "endTime": "2025-07-25T07:30:00.000",
  "isMainSleep": true,
  "levels": {
    "summary": {
      "rem": {
        "count": 10,
        "minutes": 35,
        "thirtyDayAvgMinutes": 40
      }
      ...
    }
  }
}

So I also needed a way to define which fields that I wanted to let the user extract from the integration. For this I’m using the JSONPath query language, a quite widely adopted language for extracting information from JSON data. It allows you to write strings like $.levels.summary.rem.minutes to extract the total minutes of REM sleep from the output JSON:

{
  "fields": {
    "efficiency": {
      "name": "Efficiency",
      "path": "$.efficiency"
    },
    "deep_sleep": {
      "name": "Deep sleep minutes",
      "path": "$.levels.summary.deep.minutes"
    },
    "light_sleep": {
      "name": "Light sleep minutes",
      "path": "$.levels.summary.light.minutes"
    },
    "rem_sleep": {
      "name": "REM sleep minutes",
      "path": "$.levels.summary.rem.minutes"
    },
    "minutes_asleep": {
      "name": "Minutes asleep",
      "path": "$.minutesAsleep"
    },
    "minutes_awake": {
      "name": "Minutes awake",
      "path": "$.minutesAwake"
    }
  }
}

There are a bunch of implementations available for Golang, but I settled on github.com/PaesslerAG/jsonpath since it seemed the most maintained and robust. Extracting a field is done as easily as the following:

value, err := jsonpath.Get("$.levels.summary.rem.minutes", data)

...

Here, data is a map[string]any which JSON can be easily unmarshalled to with the standard encoding/json package.

Authorization #

In order to actually fetch the data from the remote integration, the user must of course authorize our application. One of the most widely known and adopted standards for this is OAuth, which stands for “Open Authorization”. The basic principle is this:

We tell the 3rd party which scopes we are interested in reading, such as heartrate, sleep, activity and send the user off to login and accept our application. After successful authorization, the user is redirected back to our page with a code such as https://ourbackend.com/callback?code=416oXqL1Sh0EPmOjSl5fgOZXEsE2vu16SO.

This authorization code can be exchanged for an access token and refresh token. The access token is used for subsequent API calls such as https://api.fitbit.com/1.2/user/-/sleep/date/2014-06-03.json. It is also short-lived for security reasons, that’s where the refresh token comes in to give birth to a new one.

This authorization flow might sound like a scary endeavour to implement all by ourselves, but luckily we have the amazing library https://github.com/golang/oauth2 to rely on!

All we have to do is pass in a bunch of options and lean back…

conf := &oauth2.Config{
	ClientID:     "YOUR_CLIENT_ID",
	ClientSecret: "YOUR_CLIENT_SECRET",
	Scopes:       []string{"SCOPE1", "SCOPE2"},
	Endpoint: oauth2.Endpoint{
		AuthURL:  "https://provider.com/o/oauth2/auth",
		TokenURL: "https://provider.com/o/oauth2/token",
	},
}

This helps keep our system extensible and dynamic, because we can put these options together with the rest of the settings in the JSON definition of our integration.

Using the library #

By using conf.AuthCodeURL() we can generate the URL to the remote login page, where the user can authorize our application. After successful authorization, the user is redirected back to our page with a code that we can feed to conf.Exchange to generate our access and refresh token pair.

What’s even more beautiful is that we can seamlessly create a http.Client with conf.Client(). This will be authenticated for any subsequent request AND automatically refresh our token pairs when they’ve expired. Who said we had to do any hard work?

The only missing piece is that we have no way of knowing when the token was magically refreshed, thus we cannot update the credentials in our database. Luckily, oauth2 allows us to configure a custom implementation of the TokenSource interface. By checking token.AccessToken != l.current.AccessToken we will know if our token was given a new value and thus was refreshed.

Scheduling jobs #

Another important aspect of this system is that we want to periodically fetch data from these 3rd parties in order to stay up-to-date. Frequency is also contrasting for different types of data, Todoist’s completed tasks for example change much more frequently than Fitbit’s sleep. How do we create a flexible and extensible system for defining at which interval to fetch data?

No reason to reinvent the wheel, we can just use the old relic (anno 1970s), Cron! What we use is in fact not cron itself, but the syntax used for configuring it. It looks something like this:

I found this a bit confusing to understand at first, but it’s surprisingly powerful once you know how to use it. Let me charm you with an example:

Schedule at 4:30 PM every Monday, Wednesday, Friday becomes
30 4 * * 1,3,5

The ⭐ here signals that any value is accepted, i.e on any day of the month, every month. This is actually a rather complicated example compared to our use case, most of our intervals look more like */5 * * * * which simply means “run every 5 minutes”.

For implementing the cron syntax I used the wonderful library https://github.com/go-co-op/gocron. Using it is straightforward:

s, _ := gocron.NewScheduler()

job, _ = s.NewJob(
	gocron.CronJob(
		"*/5 * * * *",
		false,
	),
	gocron.NewTask(
		func() {
			// In here we run the fetch logic	
		},
	),
)

Simply pass in the cron string and a callback function to run whenever the interval is reached. The returned job can also be deleted easily if the user were to delete their integration.

Ideas for improvement / What’s next? #

I must admit that the fault handling is very minimal in this implementation. If the 3rd party is undergoing maintenance for example, the user will suddenly have missing data in their dashboard. An improvement would be to implement more resilient error handling, such as retrying the fetch after a certain amount of time.

Another thing that is missing is reversing the direction of data, what if we have a remote service wanting to send data to us? A webhook system or exposing an API endpoint would be a great addition.