Extract Microsoft To Do steps/sub-tasks from your web browser (with Asana import example)

Hey. Are you stuck in the year 2020?

Are you trying backup Microsoft To Do tasks with PowerShell and Microsoft Graph?

Have you become infuriated because the Graph API doesn’t let you export sub-tasks?

Well, the good news is that in 2022 – in the future – Microsoft Graph will add support for what they call a checklistItem resource type, which you could very possibly add to the PowerShell solution.

But for the rest of this blog post, let’s pretend that’s not the case, because there is a neat way of extracting all your Microsoft To Do data from your very own web browser.

So stop washing that orange, pick up the nearest fidget spinner, and let’s get crocing…

Index success

When you open To Do in your web browser, you see all of your lists, tasks, sub-tasks, and so on. If you disconnect from the internet, you can still click through everything. So “the stuff” must be somewhere on your device. But what is the stuff and where does it live?

Microsoft To Do is a single-page application and it stores its data in IndexedDB, a client-side data store built into modern web browsers. Here’s what my To Do database looks like in Chrome’s developer tools:

And here’s an example from my lists, the somewhat-useful “List o list”:

You see that the list has a unique identifier – id. As you might hope, a task has a field called list_id, which links it to its parent. A task has an id of its own, which is referenced by a step (sub-task) as – you guessed it – task_id. It’s IDs all the way and the data we need is clearly there. But how do we get it?

Requesting extraction

Thankfully, Florian Reuschel has written a fantastic code snippet that lets you dump an IndexedDB database to JSON in your dev tools console. I’ve put some step-by-step instructions on GitHub here.

The JSON file on its own serves as a form of backup – all of your data is present, even if it’s not in an immediately useful format.

I’ve also put a Python script on GitHub that converts the JSON file to Asana’s CSV Importer format.

It was a “write once, run once” kind of thing, made for someone who got in touch via my blog (hi Charlie!). It’s not a masterpiece of modular software design and it has to deal with some quirks in Asana’s API. But I think it’s fairly easy to reason with and hopefully can serve as inspiration if you need to do something similar.

Does it even work?

Yes!

From this:

The OG “List o list”

Via this:

Dumping IndexedDB from browser dev console
JSON to Asana CSV conversion in Python

To this:

“List o list” in Asana land, with sub-tasks included

The process relies on undocumented behaviour, so a solution using the updated Graph API would strictly be more correct (but less fun).

However, at the time of writing, the whole thing works suprisingly well, so if you do want to migrate from Microsoft To Do from Asana it might be worth trying. As ever, make sure you test with some dummy data/test accounts before you commit to anything permanent.

The code and instructions are over here on GitHub.

Backup / migrate Microsoft To Do tasks with PowerShell and Microsoft Graph

Update: this post has a spiritual successor – Extract Microsoft To Do steps/sub-tasks from your web browser (with Asana import example)

For more than a year, I’ve foolishly been using a developer Office 365 subscription for some personal stuff. You know, the subscription where they delete your data if “development activity” isn’t detected every few months. As such, I’ve periodically had to fake some development activity in order to keep the clock ticking.

Not a sustainable situation, and it’s time to sort it out…

For me, this involves moving data from one subscription’s OneDrive to another. I’m fairly confident that Rclone will be able to handle this – it’s an excellent bit of software.

It also means moving Microsoft To Do tasks between subscriptions. Ah.

Not so easy

I couldn’t find an easy way of backing up To Do. There is mention of an Outlook backup option in the docs, but it’s missing on my account. And To Do will happily suck in data from Wunderlist but I can’t see an equivalent to get data out. Where’s the Justice Department when you need them?

Luckily Microsoft Graph has a To Do API in preview and I was able to put together a script to do the lifting for me.

Ironically, this has involved an intense burst of real developer activity…

Enter the Dragon

The full script is over on GitHub.

It provides two functions:

  • Export-MicrosoftTodo saves every Microsoft To Do list and task to an XML file.
  • Import-MicrosoftTodo loads this XML file and restores all lists and tasks.

This is what a backup looks like:

Here’s a restore:

And this is what the client sees:

You can see that the completed status of the tasks has been copied. This is also true of created/modified dates, reminders, notes, and so on.

The script is quite long, so I won’t paste the whole thing here, but here are a few interesting bits:

Emotional support

Thankfully, for the comfort of our technology-addled minds, Microsoft To Do lets you decorate your lists with little emojis. Internally, it looks like if the first character of the name is an emoji, it gets special treatment in the UI.

I was having difficulty creating these special list names but the fix was simply to add charset=utf-8 to Invoke-RestMethod‘s ContentType:

$params = @{
    "Method"         = "Post"
    "Uri"            = ($graphBaseUri + "/me/todo/lists")
    "Authentication" = "OAuth"
    "Token"          = $accessToken
    "Body"           = @{
        "displayName" = $list.displayName
    } | ConvertTo-Json
    # utf-8 makes emojis work. Life priorities are correct.
    "ContentType"    = "application/json; charset=utf-8"
}
Invoke-RestMethod @params

Before and after:

Jason Bateman

The basic aim of the script is to retrieve data from one API endpoint and to later submit the same data to another endpoint.

I found that PowerShell’s – er – adorable magic got in the way a bit. Specifically, the JSON (de-)serialisation done by Convert*-Json / Invoke-RestMethod didn’t preserve empty properties, and the conversion to/from a DateTime object didn’t match the format expected by the API – and I couldn’t see an easy way to override this behaviour.

My solution was to use an alternative JSON parser available in .NET to grab the appropriate bit of the HTTP response, remove a few properties, and store the resulting JSON as a string, to be later POSTed back to the API verbatim:

$response = Invoke-WebRequest -Uri $uri -Authentication OAuth -Token $accessToken
# Invoke-RestMethod / ConvertFrom-Json mangles the response, which I resent,
# so we're using an alternative parser and storing the JSON as a string
# https://stackoverflow.com/a/58169326/12055271
$json = [Newtonsoft.Json.JsonConvert]::DeserializeObject($response.Content)
ForEach ($task in $json.value) {
    # Don't need ID - Graph API can generate a new one
    $task.Remove("id") | Out-Null
    # Don't need ETag
    $task.Remove("@odata.etag") | Out-Null
    $results += $task.ToString()
}

I also chose to save the exported data to disk using PowerShell’s CLI XML format – rather than JSON – as an easy way of guarantee the string stays as it is.

Token effort

The script needs an OAuth2 token in order to authenticate with your Microsoft account.

An easy way to get going (and slightly hacky but fine for personal use) is to grant yourself all Tasks.* permissions in Graph Explorer and copy its token, as demoed here:

(Thanks GoToGuy for this blog post.)

Please read the following license agreement carefully

A few notes on the design of the script:

  • It worked well for me and 5000(!) tasks, but please do your own testing. You can create a test Microsoft account with a secondary email address, or make an Azure tenant for free.
  • Tested with PowerShell 7 only. Get with the times.
  • Export-MicrosoftTodo currently backs up every task and Import-MicrosoftTodo restores every task.
  • If you run Import-MicrosoftTodo twice you’ll end up with duplicates.
  • The account used for export/import is the one that generated the OAuth token. You can backup from one account and restore to another simply by providing different tokens.
  • The script does not currently migrate linkedResources – these “represent[…] an item in a partner application related to a todoTask.” Shrug.
  • Nor does it share any lists as part of data import.
  • Currently, the script needs to be run interactively, in order to receive the OAuth token and to confirm a restore.
  • I’d be open to making improvements in these areas if there’s interest! The script could backup individual lists, for example, or backup someone else’s account (with the appropriate permissions).
  • Unfortunately, I don’t think there’s currently any way to retain list groups.

And in conclusion

Thanks for reading!

This has been a fun project and hopefully you can get some use out of the methods used or the script itself.