Lucid Card Integrations

Lucidspark allows rich data-driven integrations for task management systems. A Lucid card integration can:

Import tasks from a task management system and add them as cards on a Lucidspark board.
Create new cards on a Lucidspark board that sync back to the task management system as new tasks.
Convert existing shapes, like sticky notes, to cards that sync back to the task management system as new tasks.
Almost instantaneously sync changes made to data on Lucidspark cards back to the task management system.
Almost instantaneously sync changes made in the task management system to any connected Lucidspark board.

Building a Lucid card integration requires the following steps:

In an editor extension, extend LucidCardIntegration and implement the functionality you want to support.
Define an OAuth provider in your package manifest to allow API calls to the data source.
Build a publicly-routable data connector that can read and write from the data source, and include configuration for it in your package manifest.

This section of the developer guide will walk you through building a Lucid card integration to visualize Todoist tasks.

Goal

By the end of this guide you will have created a Lucid cards Todoist integration that allows you to import your Todoist tasks into Lucidspark.

Prerequisites

Make sure that you have the latest versions of lucid-package and lucid-extension-sdk npm modules installed, and you have access to the developer portal. Check out the Getting started section for details.

Step by Step Walkthrough

Create the application

Run the following commands to create a new application and set up an editor extension within that application:

npx lucid-package create todoist
cd todoist
npx lucid-package create-editor-extension todoist

Extend LucidCardIntegration

In the folder /editorextensions/todoist/src create a new file todoistcardintegration.ts. In this file, create a new class that extends LucidCardIntegration:

import {LucidCardIntegration, EditorClient} from 'lucid-extension-sdk';

export class TodoistCardIntegration extends LucidCardIntegration {
    constructor(private readonly editorClient: EditorClient) {
           super(editorClient);
    }

    // You will fill this in soon
}

Now that you have created a new Lucid card integration, you need to construct and register it. Replace the code in extension.ts with the following:

import {EditorClient, LucidCardIntegrationRegistry} from 'lucid-extension-sdk';
import {TodoistCardIntegration} from './todoistcardintegration';

const client = new EditorClient();
LucidCardIntegrationRegistry.addCardIntegration(client, new TodoistCardIntegration(client));

Initialize constants

A functional Lucid card integration depends on two individual components, the editor extension and the data connector, and it is essential to ensure proper communication between these two components. In particular, you will need to make sure that the schema used to represent the data in the extension and the data connector remain in sync.

Create a new folder called common in the root of your application, then create a new file called names.ts which will store keys which will be used by both the data connector, and the editor extension:

export const DataConnectorName = 'todoist';
export const CollectionName = 'Tasks'; // You will use this later to store the data you get from Todoist

// You will add more constants here in the future

Basic configuration

Now you can begin configuring your Lucid card integration. These fields are required for every Lucid card integration:

label: the name of the integration, e.g. "Todoist".
itemLabel: what to call one data item, e.g. "Task" or "Todoist task".
itemsLabel: what to call multiple data items, e.g. "Tasks" or "Todoist tasks".
iconUrl: a URL (a data URI is fine) pointing to an icon representing the integration.
This will be displayed at up to 32x32 CSS pixels in size.
dataConnectorName: the name of the data connector for this card integration. This will be specified in the package manifest.

We also allow you to optionally configure some more fields that let you customize your card integration:

textStyle: styles to be applied to all text on any cards created by this integration.

export class TodoistCardIntegration extends LucidCardIntegration {
    // ...

    public label = 'Todoist';
    public itemLabel = 'Todoist task';
    public itemsLabel = 'Todoist tasks';
    public iconUrl = 'https://cdn-cashy-static-assets.lucidchart.com/extensibility/packages/todoist/Todoist-logo-for-card.svg';
    public dataConnectorName: string = DataConnectorName; // Imported from names.ts that was created above
}

Field configuration

Many task management systems can have hundreds of fields, and it may be impractical or not useful to import all of them. To that end, card integrations allow the developer to configure which fields should be imported.

Define all the fields you want to import from Todoist in names.ts:

// ...

export enum DefaultFieldNames {
    Id = 'Id',
    Content = 'Content',
    Description = 'Description',
    Completed = 'Completed',
    Due = 'Due Date',
    Link = 'Link',
    Project = 'Project',
}

Every card integration must provide a fieldConfiguration object with the following members:

getAllFields - Given a data source already imported on this document by this card integration, return a list of all fields that could be eligible for import. The data source is provided because the list of fields available in case producing the list of available fields is easier with access to the imported data.

onSelectedFieldsChange - Optionally, you may provide a callback for code you want to run after the user adjusts which fields should be imported. This is usually an empty function call, as data will automatically be re-requested from your data connector if the list of requested fields changes. Not providing a value for this field will prevent users from configuring the fields shown in the card details panel:

export class TodoistCardIntegration extends LucidCardIntegration {
    // ...

    public fieldConfiguration = {
        getAllFields: (dataSource: DataSourceProxy) => {
            return Promise.resolve([...Object.values(DefaultFieldNames)]);
        },
        onSelectedFieldsChange: async (dataSource: DataSourceProxy, selectedFields: string[]) => {},
    };
}

Card shape configuration

Lucid card integrations must provide the default configuration to use for the appearance of cards imported or created by the integration.

As with the getAllFields callback described above, the getDefaultConfig callback accepts a DataSourceProxy as a parameter, indicating which data connection the configuration is for, in case you need access to the imported data to determine the list of fields to display.

The following information can be provided:

cardConfig.fieldNames: an array of field names that should be displayed as text fields on the card shapes on-canvas
cardConfig.fieldStyles: optionally, text style overrides that should be applied for specific fields, on top of the default textStyle described in Basic configuration.
cardConfig.fieldDisplaySettings: information about fields that should be displayed as data graphics on card shapes
cardDetailsPanelConfig.fields: an array of fields that has been imported and should be shown in the card details panel, with a name and optionally a locked boolean indicating whether the field cannot be unchecked from the list of fields to import

The following example configures the Todoist Lucid card to display the data from the content field on the Todoist task as text. Furthermore, the ID and due date of the task will be displayed as badges on the card. The content, description and due date of the task will also be shown in the card details panel:

export class TodoistCardIntegration extends LucidCardIntegration {
    // ...

    public getDefaultConfig = (dataSource: DataSourceProxy) => {
        return Promise.resolve({
            cardConfig: {
                fieldNames: [DefaultFieldNames.Content],
                fieldDisplaySettings: new Map([
                    [
                        DefaultFieldNames.Id,
                        {
                            stencilConfig: {
                                displayType: FieldDisplayType.SquareImageBadge,
                                valueFormula:
                                    '="https://cdn-cashy-static-assets.lucidchart.com/extensibility/packages/todoist/Todoist-logo-for-card.svg"',
                                onClickHandlerKey: OnClickHandlerKeys.OpenBrowserWindow,
                                linkFormula: '=@Link',
                                horizontalPosition: HorizontalBadgePos.RIGHT,
                                tooltipFormula:
                                    '=IF(ISNOTEMPTY(LASTSYNCTIME), "Last synced " & RELATIVETIMEFORMAT(LASTSYNCTIME), "Open in Todoist")',
                                backgroundColor: '#00000000',
                            },
                        },
                    ],
                    [
                        DefaultFieldNames.Due,
                        {
                            stencilConfig: {
                                displayType: FieldDisplayType.DateBadge,
                                tooltipFormula: '=@"Due Date"',
                            },
                        },
                    ],
                ]),
            },
            cardDetailsPanelConfig: {
                fields: [
                    {
                        name: DefaultFieldNames.Content,
                        locked: true,
                    },
                    {
                        name: DefaultFieldNames.Description,
                        locked: true,
                    },
                    {
                        name: DefaultFieldNames.Due,
                        locked: true,
                    },
{
                        name: DefaultFieldNames.Completed,
                        locked: true,
                    }
                ],
            },
        });
    };
}

Field display settings

For fields included in fieldDisplaySettings, a variety of display options are available by selecting a value for stencilConfig.displayType.

In each case, only the displayType is strictly required. The other options that you can set on these display configuration objects are as follows:

backgroundColor

For displayType values that support it, you may provide an override for the background color for this data.
If this is not provided, the background color will default to the background color of the card itself, darkened 5%.

This may be provided as a literal color hex, e.g. "#00ff00ff" or as a formula by starting the string with "=", e.g. "=darken("#ffffff", 0.5)"

valueFormula

If specified, the result of this formula (executed in the context of the data item associated with this card) will be used instead of the raw field value when creating the data graphic. This can be useful if, for example, you want to convert an ID into a URL.

tooltipFormula

If specified, the result of this formula (executed in the context of the data item associated with this card) will be used as a tooltip when the user hovers the cursor over this data graphic.

horizontalPosition

Each display type has its default location on the card. You can override those default locations by setting these values.

verticalPosition

Each display type has its default location on the card. You can override those default locations by setting these values.

onClickHandlerKey

If specified, what behavior should happen when the user clicks on the data graphic generated via the above displayType?

linkFormula

If onClickHandlerKey is OpenBrowserWindow, this formula calculates the URL to open.

Create the import modal

Most Lucid card integrations will allow users to import data from their data source using the standard card import modal. To configure that modal for search and import, specify the importModal member of your LucidCardIntegration. You will provide three asynchronous callbacks that specify the set of search fields the user can fill out, perform the search based on those values, and finally import the data selected as new cards on-canvas. There is a fourth optional callback that can be used on the setup process.

In the /editorextensions/todoist/src/ directory, create a new file todoistimportmodal.ts. Here you will create a class that configures the import modal for the Todoist card integration:

import {
    CollectionDefinition,
    CollectionProxy,
    EditorClient,
    ExtensionCardFieldDefinition,
    SerializedFieldType,
} from 'lucid-extension-sdk';

export class TodoistImportModal {
    constructor(private readonly editorClient: EditorClient) {}

    // Stub these out for now

    public async getSearchFields(
        searchSoFar: Map<string, SerializedFieldType>,
    ): Promise<ExtensionCardFieldDefinition[]> {
        throw 'Not implemented';
    }

    public async search(fields: Map<string, SerializedFieldType>): Promise<{
        data: CollectionDefinition;
        fields: ExtensionCardFieldDefinition[];
        partialImportMetadata: {collectionId: string; syncDataSourceId?: string};
    }> {
        throw 'Not implemented';
    }

    public async import(
        primaryKeys: string[],
        searchFields: Map<string, SerializedFieldType>,
    ): Promise<{collection: CollectionProxy; primaryKeys: string[]}> {
        throw 'Not implemented';
    }
}

Now, construct this class and assign it to the import field of the TodoistCardIntegration:

export class TodoistCardIntegration extends LucidCardIntegration {
    // ...

    public importModal = new TodoistImportModal(this.editorClient);
}

Run the integration

This is a good time to check if everything is working as intended. In the root of you application, execute the following command to run your extension:

npx lucid-package test-editor-extension todoist

Now open a Lucidspark board and enable the Load local extensions option in the developer menu. Checking this option will automatically refresh the page and have the document attempt to load your locally running extension.

You should now see the Todoist integration show up in the submenu that is shown when you click on the more tools button (ellipses icon) located at the bottom of the left toolbar. Click the pin icon on the right to pin the Todoist integration to your toolbar.

You can now trigger the import modal. There's an error displayed because the search callback just throws an error.

Partial Import

Connecting to data

Now that you have the import modal showing up, you will need to fetch data from Todoist's OAuth 2.0 REST API to populate the import modal.

Configuring OAuth

The Lucid extensibility platform allows extensions to authorize with a third party using OAuth 2.0. To do so, you will first need to generate a unique identifier for your app.

Navigate to developer portal and create a new application. Once created, copy the app ID from the URL and paste it in the id field of your application's manifest.json file.

Copy Package ID

Now fill in the OAuth 2.0 provider details in the oauthProviders section of the manifest.json file. These provider details can be found in Todoist's API documentation:

{
    // ...
    "oauthProviders": [
        {
          "name": "todoist",
          "title": "Todoist",
          "authorizationUrl": "https://todoist.com/oauth/authorize",
          "tokenUrl": "https://todoist.com/oauth/access_token",
          "scopes": ["data:read_write"],
          "domainWhitelist": ["https://api.todoist.com", "https://todoist.com"],
          "clientAuthentication": "clientParameters"
        }
    ]
}

Create the OAuth client

Before you can start making API requests to Todoist, you will also need to set up an OAuth 2.0 client in Todoist. Navigate to the Todoist developer console and create a new app.

Create Oauth

Once created, you will need to provide the OAuth redirect URL to Todoist. This is the URL users will be redirected to after granting your application access to their Todoist instance. Since Lucid handles OAuth 2.0 authentication for your app, a URL of the following form will need to be provided:

https://extensibility.lucid.app/packages/<applicationId>/oauthProviders/<oauthProviderName>/authorized

You will need to replace applicationId with the ID present in the id field of your manifest.json file and oauthProviderName with the name you gave your OAuth provider in the manifest.json file. For this guide, the oauthProviderName was set to be todoist.

Once you've entered the redirect URL in Todoist, click save settings.

Give your app the client credentials

Now that you have the client set up, you will need to give your app the appropriate credentials so that it can communicate with the Todoist client.

Create a new file <oauthProviderName>.credentials.local (for this guide, this file would be named todoist.credentials.local) in the root directory of your application (where the manifest.json file exists). Add a JSON object containing the OAuth client ID and client secret to this file. The ID and secret can be found in the app dashboard in Todoist:

{
    "clientId": "Client ID for your OAuth provider",
    "clientSecret": "Client secret for your OAuth provider"
}

Create the Todoist client

With OAuth configured you can now begin making requests to Todoist to populate the import modal.

In /editorextensions/todoist/src create a new folder named model and add a new file todoistmodel.ts to it. This file will contain interfaces representing the data you get from Todoist:

export interface Project {
    id: string;
    name: string;
    comment_count: number;
}

export interface TodoistDate {
    date: string;
    is_recurring: boolean;
    datetime: string;
    string: string;
    timezone: string;
}

export interface Task {
    id: string;
    is_completed: boolean;
    content: string;
    description: string;
    due: TodoistDate;
    url: string;
}

In /editorextensions/todoist/src create another folder named net and add a new file todoistclient.ts. This will contain a class that will be responsible for making the API requests to Todoist:

import {EditorClient, HumanReadableError, XHRResponse} from 'lucid-extension-sdk';
import { Project, Task } from '../model/todoistmodel';

export class TodoistClient {
    constructor(private readonly client: EditorClient) {}

    private readonly todoistOAuthProviderName = 'todoist';
    private readonly baseUrl = 'https://api.todoist.com/rest/v2/'

    public async getProjects(): Promise<Project[]> {
        const rawResponse = await this.makeGetRequest(`${this.baseUrl}projects`);
        return this.parseAsAny(rawResponse) as Project[];
    }

    public async getTasks(projectId: string, search?: string): Promise<Task[]> {
        const filterParam = search ? encodeURI(`&filter=search:${search}`) : '';
        const rawResponse = await this.makeGetRequest(`${this.baseUrl}tasks?project_id=${projectId}${filterParam}`);
        return this.parseAsAny(rawResponse) as Task[];
    }

    private async makeGetRequest(url: string) {
        try {
            const response = await this.client.oauthXhr(this.todoistOAuthProviderName, {
                url,
                method: 'GET',
                responseFormat: 'utf8',
            });
            return response;
        } catch (error) {
            console.log('Error:', error);
            throw this.errorFromResponse(error);
        }
    }

    private errorFromResponse(response: any) {
        try {
            return new HumanReadableError(this.parseAsAny(response).errors[0].message);
        } catch (error) {
            return new Error(JSON.stringify(response));
        }
    }

    private parseAsAny(data: XHRResponse): any {
        switch (data.responseFormat) {
            case 'utf8':
                return JSON.parse(data.responseText) as any;
            case 'binary':
                return JSON.parse(data.responseData as any) as any;
        }
    }
}

Finally, create an instance of this client class in TodoistImportModal:

export class TodoistImportModal {
    private todoistClient: TodoistClient;

    constructor(private readonly editorClient: EditorClient) {
        this.todoistClient = new TodoistClient(editorClient);
    }

    // ...
}

Populate the import modal

You can now begin implementing the callbacks in the TodoistImportModal class.

getSearchFields callback

The getSearchFields callback returns a list of fields to display on the search modal, which the user selects to search or filter the available data. The callback takes all the entered search field values so far as a parameter, in case you want to return different search fields based on values entered in the form so far (e.g. to offer a dropdown to select a project based on the workspace already selected).

The following example displays a simple text search box, plus a dropdown to select what project they would like to see tasks from:

export class TodoistImportModal {
    // ...

    private readonly searchField = 'search';
    private readonly projectField = 'project';

    public async getSearchFields(
        searchSoFar: Map<string, SerializedFieldType>,
    ): Promise<ExtensionCardFieldDefinition[]> {
        const projects = await this.todoistClient.getProjects();

        const fields: ExtensionCardFieldDefinition[] = [
            {
                name: this.searchField,
                label: 'Search',
                type: ScalarFieldTypeEnum.STRING,
            },
            {
                name: this.projectField,
                label: 'Project',
                type: ScalarFieldTypeEnum.STRING,
                default: projects[0]?.id,
                constraints: [{type: FieldConstraintType.REQUIRED}],
                options: projects.map((project) => ({
                    label: project.name,
                    value: project.id,
                })),
            },
        ];

        return fields;
    }

    // ...
}

Supported search field types

While this integration will only be using the text field and a single select dropdown during import, Lucid supports several other types of user input fields.

The following examples are all currently-supported search field types:

{
    name: 'search',
    label: 'Search',
    type: ScalarFieldTypeEnum.STRING,
}

{
    name: 'complete',
    label: 'Status',

    //Note: May be NUMBER or STRING as well, depending on the "value" set on options below
    type: ScalarFieldTypeEnum.BOOLEAN,

    constraints: [{type: FieldConstraintType.MAX_VALUE, value: 1}],
    options: [
        {label: 'Completed', value: true},
        {label: 'Not Completed', value: false},
    ],
}

{
    name: 'complete',
    label: 'Status',

    //Note: May be NUMBER or STRING as well, depending on the "value" set on options below
    type: ScalarFieldTypeEnum.BOOLEAN,

    options: [
        {label: 'Completed', value: true},
        {label: 'Not Completed', value: false},
    ],
}

📘
Use the MAX_VALUE constraint to specify single-select or multi-select dropdown, as above.

const optionsCallback = LucidCardIntegrationRegistry.registerFieldOptionsCallback(client, async () => {
    return [
        {label: 'Completed', value: true},
        {label: 'Not Completed', value: false},
    ];
});

// ...

{
    name: 'complete',
    label: 'Status',
    type: ScalarFieldTypeEnum.BOOLEAN,
    options: optionsCallback,
}

const searchCallback = LucidCardIntegrationRegistry.registerFieldSearchCallback(client, async (searchText) => {
    return [
        {label: 'Tabitha Ross', value:'Tabitha Ross'},
        {label: 'Stanley Browning', value:'Stanley Browning'},
        {label: 'Randall Lucas', value:'Randall Lucas'},
        {label: 'Kira Ellis', value:'Kira Ellis'},
        {label: 'Dale Bauer', value:'Dale Bauer'},
        {label: 'Itzel Knight', value:'Itzel Knight'},
        {label: 'Sage Beltran', value:'Sage Beltran'},
        {label: 'Iris Ponce', value:'Iris Ponce'},
        {label: 'Gisselle Conway', value:'Gisselle Conway'},
        {label: 'Emely Williams', value:'Emely Williams'},
        {label: 'Elena Arias', value:'Elena Arias'},
        {label: 'Sarahi Aguirre', value:'Sarahi Aguirre'},
    ].filter(one => one.label.includes(searchText));
}),

// ...

{
    name: 'owner',
    label: 'Owner',
    type: ScalarFieldTypeEnum.STRING,
    search: searchCallback,
}

search callback

The next step is to implement the actual search functionality based on the values the user has entered into the search fields you specified above.

The search callback returns a CollectionDefinition including a schema and data items representing the search results. It also returns a list of fields to display as columns in the user-visible data table in the search modal.

In the example below, requests are being made to Todoist that search tasks belonging to the specified project. It then uses the response to populate the import modal:

export class TodoistImportModal {
    // ...

    public async search(fields: Map<string, SerializedFieldType>): Promise<{
        data: CollectionDefinition;
        fields: ExtensionCardFieldDefinition[];
        partialImportMetadata: {collectionId: string; syncDataSourceId?: string};
    }> {
        let search = fields.get(this.searchField);
        if (!isString(search)) {
            search = '';
        }

        const projectId = fields.get(this.projectField) as string | undefined;

        const tasks = projectId ? await this.todoistClient.getTasks(projectId, search) : [];

        return {
            data: {
                schema: {
                    fields: [
                        {
                            name: DefaultFieldNames.Id,
                            type: ScalarFieldTypeEnum.STRING,
                        },
                        {
                            name: DefaultFieldNames.Content,
                            type: ScalarFieldTypeEnum.STRING,
                            mapping: [SemanticFields.Title],
                        },
                        {
                            name: DefaultFieldNames.Completed,
                            type: ScalarFieldTypeEnum.BOOLEAN,
                        },
                        {
                            name: DefaultFieldNames.Due,
                            type: ScalarFieldTypeEnum.DATEONLY,
                            mapping: [SemanticFields.Time],
                        },
                    ],
                    primaryKey: [DefaultFieldNames.Id],
                },
                items: new Map(
                    tasks.map((task) => [
                        JSON.stringify(task.id),
                        {
                            [DefaultFieldNames.Id]: task.id,
                            [DefaultFieldNames.Content]: task.content,
                            [DefaultFieldNames.Completed]: task.is_completed,
                            [DefaultFieldNames.Due]: task.due
                                ? {ms: Date.parse(task.due.date), isDateOnly: true}
                                : undefined,
                        },
                    ]),
                ),
            },
            fields: [
                {
                    name: DefaultFieldNames.Content,
                    label: DefaultFieldNames.Content,
                    type: ScalarFieldTypeEnum.STRING,
                },
                {
                    name: DefaultFieldNames.Completed,
                    label: DefaultFieldNames.Completed,
                    type: ScalarFieldTypeEnum.BOOLEAN,
                },
                {
                    name: DefaultFieldNames.Due,
                    label: 'Due',
                    type: ScalarFieldTypeEnum.STRING,
                },
            ],
            partialImportMetadata: {
                collectionId: CollectionName,
                syncDataSourceId: projectId,
            },
        };
    }

    // ...
}

The optional partialImportMetadata that you can return from the search callback is used to prefill the Lucid task card with data while an import is being performed in the background.

onSetup callback

During the import modal setup, if you want to add any custom setup code, you can do by implementing this optional callback.

It is going to be called everytime right after the modal is displayed and before the first call to getSearchFields.

Run the integration

At this point you can run the integration to test whether your import modal is functioning properly.
Make sure to add some tasks to your Todoist account!

Note: If you have the extension running locally you do not need to restart it when you make changes to the code. The local development server will refresh automatically when changes are detected. All you need to do is refresh the Lucidspark board to get the newest version of your extension.

Import Modal

Import data as Lucid cards

Now that you've allowed the user to find the tasks they want to import, you need to provide the callback that actually imports the selected tasks.

The import for this card integration will be performed with the help of a data-connector. A data-connector is run on its own server independent of the Lucid extension. The extension will make a request to the data connector to perform an import and pass in the required information. It is the data-connector's responsibility to fetch the data from the external source, convert it into a format Lucid understands and send a request to Lucid containing the data to be imported.

Note: To make the development experience easier, the Lucid sdk offers a debug server that can be used for local development. This guide will walk you through using the debug server. However, to make the integration publicly available, you will have to host the data-connector and provide a public URL to it in the manifest.

Create the data-connector

In the root of you application run the following command to create a new data-connector:

npx lucid-package create-data-connector todoist

This will create a new directory named dataconnectors containing a folder named todoist. It will also create an entry for the data-connector in the manifest.json file.

To provide the data-connector details to the extension, in the manifest.json file, fill in the fields in the dataConnectors section:

{
    // ...

    "dataConnectors": [
        {
            "name": "todoist",
            "oauthProviderName": "todoist",
            "callbackBaseUrl": "http://localhost:3001?kind=action&name=",
            "dataActions": {
                "Import": "Import"
            }
        }
    ]
}

This declares a data-connector named todoist that uses the OAuth token for the todoist OAuth provider to make requests. You will be running this data connector on localhost:3001 which is why you set the callbackBaseUrl to be http://localhost:3001?kind=action&name=. The query parameters, kind and name, are used to inform the data-connector about the type of request being received. Finally, one of the actions this data connector will perform is Import.

Add the import data action name to /common/names.ts:

// ...
export enum DataAction {
    Import = 'Import'
}

Todoist provides a typescript sdk to communicate with their API. To use this sdk in you data-connector, execute the following command in /dataconnectors/todoist/:

npm install @doist/todoist-api-typescript --save-prod

Define the schema

You will need to convert the data you get from Todoist into a format compatible with Lucid. Begin by defining this Lucid compatible format.

In /dataconnectors/todoist create a new folder collections and within this folder create a new file taskcollections.ts. In this file, declare the schema for your task collection:

import {declareSchema, FieldConstraintType, ItemType, ScalarFieldTypeEnum, SemanticKind, SerializedLucidDateObject} from 'lucid-extension-sdk';
import {DefaultFieldNames} from '../../../common/names';
import {Task as TodoistTask, TodoistApi as TodoistClient, DueDate as TodoistDueDate} from '@doist/todoist-api-typescript';

export const taskSchema = declareSchema({
    primaryKey: [DefaultFieldNames.Id],
    fields: {
        [DefaultFieldNames.Id]: {
            type: ScalarFieldTypeEnum.STRING,
            constraints: [{type: FieldConstraintType.LOCKED}],
        },
        [DefaultFieldNames.Completed]: {
            type: ScalarFieldTypeEnum.BOOLEAN,
            mapping: [SemanticFields.Status],
        },
        [DefaultFieldNames.Content]: {
            type: ScalarFieldTypeEnum.STRING,
            mapping: [SemanticFields.Title],
        },
        [DefaultFieldNames.Description]: {
            type: ScalarFieldTypeEnum.STRING,
            mapping: [SemanticFields.Description],
        },
        [DefaultFieldNames.Due]: {
            type: [ScalarFieldTypeEnum.DATE, ScalarFieldTypeEnum.DATEONLY, ScalarFieldTypeEnum.NULL] as const,
            mapping: [SemanticFields.Time],
        },
        [DefaultFieldNames.Link]: {
            type: ScalarFieldTypeEnum.STRING,
            mapping: [SemanticFields.URL],
            constraints: [{type: FieldConstraintType.LOCKED}],
        },
        [DefaultFieldNames.Project]: {
            type: ScalarFieldTypeEnum.STRING
        }
    },
});

export type TaskFieldsStructure = typeof taskSchema.example;
export type TaskItemType = ItemType<TaskFieldsStructure>;

Also add helper functions that convert the data you get from Todoist into this format:

// ...

export function getFormattedTask(task: TodoistTask): TaskItemType {
    return {
        [DefaultFieldNames.Id]: task.id,
        [DefaultFieldNames.Content]: task.content,
        [DefaultFieldNames.Description]: task.description,
        [DefaultFieldNames.Completed]: task.isCompleted,
        [DefaultFieldNames.Due]: task.due ? getFormattedDueDate(task.due) : null,
        [DefaultFieldNames.Link]: task.url,
        [DefaultFieldNames.Project]: task.projectId,
    };
}

function getFormattedDueDate(dueDate: TodoistDueDate): SerializedLucidDateObject | null {
    const dateToUse = dueDate.datetime || dueDate.date;
    if (dateToUse) {
        return {
            ms:  Date.parse(dateToUse)
        };
    }
    return null;
}

Create the import action

With the schema defined, you can now have the data-connector perform an import. In /dataconnectors/todoist/actions, add a new file import.ts:

import {TodoistApi as TodoistClient} from '@doist/todoist-api-typescript';
import {DataConnectorAsynchronousAction} from 'lucid-extension-sdk/dataconnector/actions/action';
import {CollectionName} from '../../../common/names';
import {getFormattedTask, taskSchema} from '../collections/taskcollections';

export const importAction: (action: DataConnectorAsynchronousAction) => Promise<{success: boolean}> = async (
    action,
) => {
    // action.data contains data passed to the data-connector by the extension when an action is invoked. In this integration,
    // you will have the extension pass in the ID's of the tasks the user selected in the import modal.
    const taskIds = action.data as string[];
    const todoistClient = new TodoistClient(action.context.userCredential);

    // Fetch the task data from Todoist
    const fullTaskData = await todoistClient.getTasks({ids: taskIds});

    // Convert the data into a Lucid compatible format
    const formattedTaskData = fullTaskData.map(getFormattedTask);

    // Send the imported data to Lucid
    await action.client.update({
        dataSourceName: 'Todoist',
        collections: {
            [CollectionName]: {
                schema: {
                    fields: taskSchema.array,
                    primaryKey: taskSchema.primaryKey.elements,
                },
                patch: {
                    items: taskSchema.fromItems(formattedTaskData),
                },
            },
        },
    });

    return {success: true};
};

Now, you can register the import action with the data-connector. Update /dataconnectors/todoist/index.ts with the following code:


import {DataConnector, DataConnectorClient} from 'lucid-extension-sdk';
import {DataAction} from '../../common/names';
import {importAction} from './actions/import';

export const makeDataConnector = (client: DataConnectorClient) =>
    new DataConnector(client).defineAsynchronousAction(DataAction.Import, importAction);

Run the debug server

In the dataconnectors/todoist folder execute the following commands to run the data-connector:

npx nodemon debug-server.ts

You should see the following output:

Routing / (Import)
Listening on port 3001

Call the import action

With the data-connector setup, you should now be able to update the import callback in your extension to trigger the import action.

The import callback accepts as parameters a list of primary keys selected by the user from your CollectionDefinition returned from your search callback, as well as the set of field values entered by the user in the search form (in case you need that information in order to perform the import, e.g. the user selecting which server to query).

When the callback returns, the import should be complete, and the imported data should exist in a collection on the current document:

export class TodoistImportModal {
    // ...

    public async import(
        primaryKeys: string[],
        searchFields: Map<string, SerializedFieldType>,
    ): Promise<{
        collection: CollectionProxy;
        primaryKeys: string[];
    }> {
        const projectId = searchFields.get(this.projectField);
        if (!isString(projectId) || !projectId) {
            throw new Error('No project selected');
        }

        // Call the import action
        await this.editorClient.performDataAction({
            actionName: DataAction.Import,
            syncDataSourceIdNonce: projectId,
            dataConnectorName: DataConnectorName,
            actionData: primaryKeys.map((pk) => JSON.parse(pk)), // pass in the ID's of the selected tasks to the data-connector
            asynchronous: true,
        });

        // Wait for the import to complete
        const collection = await this.editorClient.awaitDataImport(
            DataConnectorName,
            projectId,
            CollectionName,
            primaryKeys,
        );

        return {collection, primaryKeys};
    }

Perform an import

With both the editor-extension and data-connector running, you should be able to import Todoist tasks as Lucid cards.

Full Import

Fetching updates from Todoist

When linked to an external source, Lucid can update your card when changes are made outside of the document. This can be done via two different methods, hard refresh and polling, both of which are called outside of your extension when certain criteria are met.

Declaring the hard refresh and polling actions

Declare the hard refresh and polling actions by adding HardRefresh and Poll to your list of actions in the manifest.json file:

{
    //...

    "dataConnectors": [
        {
            "name": "todoist",
            "oauthProviderName": "todoist",
            "callbackBaseUrl": "http://localhost:3001?kind=action&name=",
            "dataActions": {
            "Import": "Import",
            "HardRefresh": "HardRefresh",
            "Poll": "Poll"
            }
        }
    ]
}

Then add HardRefresh and Poll to the DataAction enumeration:

...
export enum DataAction {
	Import = 'Import',
	HardRefresh = 'HardRefresh',
	Poll = 'Poll'
}

Create the hard refresh action

A hard refresh allows your extension to refresh all of the data on a document when that document is opened for the first time after it has been closed by all users for more than 5 minutes. To add hard refresh support to your data connector, add a new file named hardrefresh.ts to the /dataconnectors/todoist/actions folder and fill it in with the following code:

import {TodoistApi as TodoistClient} from '@doist/todoist-api-typescript';
import {DataConnectorAsynchronousAction} from 'lucid-extension-sdk';
import {isString} from 'lucid-extension-sdk/core/checks';
import {CollectionName} from '../../../common/names';
import {getFormattedTask, taskSchema} from '../collections/taskcollections';

export const hardRefreshAction: (action: DataConnectorAsynchronousAction) => Promise<{success: boolean}> = async (
    action
) => {
    const todoistClient = new TodoistClient(action.context.userCredential);

    // Figure out which tasks you have already imported
    let taskIds: string[] = [];
    Object.keys(action.context.documentCollections).forEach((key) => {
        if (key.includes('Tasks')) {
            taskIds = taskIds.concat(
                action.context.documentCollections?.[key].map((taskId) => JSON.parse(taskId)).filter(isString),
            );
        }
    });

    // If no tasks were imported, then you don't need to refresh
    if (taskIds.length == 0) {
        return {success: true};
    }

    // Update any tasks that you found
    const fullTaskData = await todoistClient.getTasks({ids: taskIds});
    const formattedTaskData = fullTaskData.map(getFormattedTask);

    // Send the imported data to Lucid
    await action.client.update({
        dataSourceName: 'Todoist',
        collections: {
            [CollectionName]: {
                schema: {
                    fields: taskSchema.array,
                    primaryKey: taskSchema.primaryKey.elements,
                },
                patch: {
                    items: taskSchema.fromItems(formattedTaskData),
                },
            },
        },
    });

    return {success: true};

};

Now, you can register the hard refresh action with the data connector. Update /dataconnectors/todoist/index.ts with the following code:

import {DataConnector, DataConnectorClient} from 'lucid-extension-sdk';
import {DataAction} from '../../common/names';
import {importAction} from './actions/import';
import {hardRefreshAction} from './actions/hardrefresh';

export const makeDataConnector = (client: DataConnectorClient) =>
    new DataConnector(client).defineAsynchronousAction(DataAction.Import, importAction).defineAsynchronousAction(DataAction.HardRefresh, hardRefreshAction);

Create the poll action

The poll action allows your extension to refresh all of the data on a document periodically while the document is open. In many cases, the poll action can utilize the same code as the hard refresh action:

import {DataConnector, DataConnectorClient} from 'lucid-extension-sdk';
import {DataAction} from '../../common/names';
import {importAction} from './actions/import';
import {hardRefreshAction} from './actions/hardrefresh';

export const makeDataConnector = (client: DataConnectorClient) =>
    new DataConnector(client).defineAsynchronousAction(DataAction.Import, importAction).defineAsynchronousAction(DataAction.HardRefresh, hardRefreshAction).defineAsynchronousAction(DataAction.Poll, hardRefreshAction);

Test hard refresh

With both the editor-extension and data-connector running, you should be able to test whether or not your card will refresh when opening the document. First, exit the document you have imported a task onto. Next, make some changes to the task you imported. Lastly, wait at least five minutes and open the document again. You should see the changes you made reflected in your imported card after a short period.

Test Hard Refresh

Test polling

With both the editor-extension and data-connector running, you should also be able to test whether or not your card is staying in sync through the polling action. While the document is open, make a change to your task in Todoist. Next, wait for around 30 seconds, and you should see your task update.

Test Polling

Pushing changes back to Todoist

When changes are made to Lucidcards linked to an external source, Lucid's services will automatically inform your data-connector of these changes. You can use this information to update the Task in the external data source.

Declaring the patch action

Declare the patch action by adding Patch to your list of actions in the manifest.json file:

{
    //...

    "dataConnectors": [
        {
            "name": "todoist",
            "oauthProviderName": "todoist",
            "callbackBaseUrl": "http://localhost:3001?kind=action&name=",
            "dataActions": {
            "Import": "Import",
            "Patch": "Patch"
            }
        }
    ]
}

Then add Patch to the DataAction enumeration:

...
export enum DataAction {
	Import = 'Import',
	Patch = 'Patch'
}

Implementing the patch action

First, you will need to convert the data sent by Lucid, containing changes made to the cards, to a format that can be sent to Todoist to update the task. Add the following functions to taskcolllections.ts:

import {UpdateTaskArgs as TodoistUpdateTaskArgs} from "@doist/todoist-api-typescript";

...

export function lucidPatchToTodoistPatch(
  data: Partial<TaskItemType>
): TodoistUpdateTaskArgs {
  const dueDate = data[DefaultFieldNames.Due] as
    | SerializedLucidDateObject
    | null
    | undefined;

  return {
    content: data[DefaultFieldNames.Content],
    description: data[DefaultFieldNames.Description],
    dueDatetime: dueDate ? lucidDateToTodoistDate(dueDate) : undefined,
  };
}

function lucidDateToTodoistDate(date: SerializedLucidDateObject): string {
    if ('ms' in date) {
        return new Date(date.ms).toISOString();
    }
    return new Date(date.isoDate).toISOString();
}

Now, in the actions folder, create a new file named patch.ts. This file will be responsible for handling the request containing the changes sent by Lucid. The following code parses the changes and updates the corresponding tasks in Todoist:

import {TodoistApi as TodoistClient} from '@doist/todoist-api-typescript';
import {DataConnectorPatchAction, PatchItems} from 'lucid-extension-sdk/dataconnector/actions/action';
import {PatchChange} from 'lucid-extension-sdk/dataconnector/actions/patchresponsebody';
import {DefaultFieldNames} from '../../../common/names';
import {lucidPatchToTodoistPatch, TaskItemType} from '../collections/taskcollections';

export const patchAction: (action: DataConnectorPatchAction) => Promise<PatchChange[]> = async (action) => {
    const todoistClient = new TodoistClient(action.context.userCredential);
    return await Promise.all(
        action.patches.map(async (patch) => {
            const change = patch.getChange();
            await Promise.all([updateTodoistTasks(patch.itemsChanged, change, todoistClient)]);
            return change;
        }),
    );
};

async function updateTodoistTasks(itemsChanged: PatchItems, change: PatchChange, todoistClient: TodoistClient) {
    await Promise.all(Object.entries(itemsChanged).map(async ([primaryKey, updates]) => {
        try {
            const taskId = JSON.parse(primaryKey) as string;
            if (taskIsCompleted(updates)) {
                await todoistClient.closeTask(taskId);
            } else if (taskShouldBeReOpened(updates)) {
                await todoistClient.reopenTask(taskId);
            }

            const todoistParams = await lucidPatchToTodoistPatch(updates);
            if (Object.keys(todoistParams).length > 0) {
                await todoistClient.updateTask(taskId, todoistParams);
            }
        } catch (err) {
            change.setTooltipError(primaryKey, 'Failed to update Todoist');
            console.error('error patching', err);
        }
    }));
}

function taskIsCompleted(data: Partial<TaskItemType>): boolean | undefined {
    return data[DefaultFieldNames.Completed];
}

function taskShouldBeReOpened(data: Partial<TaskItemType>): boolean {
    return data[DefaultFieldNames.Completed] === false;
}

Now, you can register the patch action with the data-connector. Update /data-connector/index.ts with the following code:

import { DataConnector, DataConnectorClient } from "lucid-extension-sdk";
import { importAction } from "./actions/import";
import { DataAction } from "../../common/names";
import { patchAction } from "./actions/patch";

export const makeDataConnector = (client: DataConnectorClient) =>
  new DataConnector(client)
    .defineAsynchronousAction(DataAction.Import, importAction)
    .defineAction(DataAction.Patch, patchAction);

Perform an update

With both the editor-extension and data-connector running, you should be able to make edits to the Lucid cards and have the corresponding task in Todoist be updated.

Update Card

Creating new tasks in Todoist

The patch action created above can also be used to create new tasks in the external data source. The patch Lucid sends to your data-connector will contain information about any new data-linked cards the user created. You can use this data to create a new task in Todoist.

Updating the patch action

The data-connector will need to convert the creation data it receives from Lucid into data that can be sent to Todoist to create the task. In taskcollections.ts add the following function:

import {AddTaskArgs as TodoistAddTaskArgs} from "@doist/todoist-api-typescript";

...

export function lucidPatchToTodoistCreationData(data: Partial<TaskItemType>): TodoistAddTaskArgs {
    const {content, description, dueDatetime} = lucidPatchToTodoistPatch(data);
    return {
        content: content ?? '', // Todoist requires this to be defined
        dueDatetime: dueDatetime ?? undefined, // The rest of these need to be undefined instead of null for creation
        description: description,
    };
}

Now, you need to add a function which creates Todoist tasks to patch.ts:

...
async function createTodoistTasks(
    itemsAdded: PatchItems,
    syncCollectionId: string,
    change: PatchChange,
    todoistClient: TodoistClient,
) {
    await Promise.all(Object.entries(itemsAdded).map(async ([oldPrimaryKey, additions]) => {
        try {
            const formattedCreate = {
                ...lucidPatchToTodoistCreationData(additions),
            };
            const taskResponse = await todoistClient.addTask(formattedCreate);
            change.collections.push({
                collectionId: syncCollectionId,
                items: {[oldPrimaryKey]: getFormattedTask(taskResponse)},
                itemsDeleted: [],
            });
        } catch (err) {
            change.setTooltipError(oldPrimaryKey, 'Failed to create task in Todoist');
            console.error('Error creating item', err);
        }
    }));
}

Finally, update the patchAction function in this file to call the task creation function you added:

...
export const patchAction: (action: DataConnectorPatchAction) => Promise<PatchChange[]> = async (action) => {
    const todoistClient = new TodoistClient(action.context.userCredential);
    return await Promise.all(
        action.patches.map(async (patch) => {
            const change = patch.getChange();
            await Promise.all([
                updateTodoistTasks(patch.itemsChanged, change, todoistClient),
                createTodoistTasks(patch.itemsAdded, patch.syncCollectionId, change, todoistClient),
            ]);
            return change;
        }),
    );
};
...

Creating the card creation callout

With the data-connector set up, you can now implement an entry point for the user to input information to create a new card. In /editorextensions/todoist/src, create a new file named todoisttaskcreator.ts:

export class TodoistTaskCreator {
    private todoistClient: TodoistClient;

    constructor(private readonly editorClient: EditorClient) {
        this.todoistClient = new TodoistClient(editorClient);
    }
}

First, you will need to define all the fields that should be shown to the user when they attempt to create a new Todoist card. The getInputFields callback handles this responsibility. The example below requires the user to provide values for the content and project field to create a new task:

export class TodoistTaskCreator {
    ...

    private readonly contentField = "content";
    private readonly projectField = "project";

    public async getInputFields(inputSoFar: Map<string, SerializedFieldType>): Promise<ExtensionCardFieldDefinition[]> {
        const projects = await this.todoistClient.getProjects();

        const fields: ExtensionCardFieldDefinition[] = [
            {
                name: this.contentField,
                label: 'Content',
                type: ScalarFieldTypeEnum.STRING,
                constraints: [{type: FieldConstraintType.REQUIRED}],
            },
            {
                name: this.projectField,
                label: 'Project',
                type: ScalarFieldTypeEnum.STRING,
                default: projects[0]?.id,
                constraints: [{type: FieldConstraintType.REQUIRED}],
                options: projects.map((project) => ({label: project.name, value: project.id})),
            },
        ];

        return fields;
    }
}

Once the user provides the information you will need to pass this information off to the data-connector so that a new task can be created in Todoist. Since updates in Lucid are automatically sent to the data-connector, all you will need to do is update the task collection with the new data. This is done in the createCardData callback:

export class TodoistTaskCreator {
    ...

    public async createCardData(
        input: Map<string, SerializedFieldType>,
    ): Promise<{collection: CollectionProxy; primaryKey: string}> {
        const projectId = input.get(this.projectField);
        if (!isString(projectId) || !projectId) {
            throw new Error('No project selected');
        }

        let collection: CollectionProxy;
        try {
// Check if the Tasks collection has already been created in the          // data source, with a 1ms timeout. If the collection doesn't exist yet,  // an exception will be thrown.
            collection = await this.editorClient.awaitDataImport(DataConnectorName, projectId, CollectionName, [], 1);
        } catch {

 	// No collection exists yet. Ask the data connector to perform an import of             // an empty list of tasks, just to get the collection created along with whatever other // metadata you might need to set up there.
            await this.editorClient.performDataAction({
                dataConnectorName: DataConnectorName,
                syncDataSourceIdNonce: projectId,
                actionName: DataAction.Import,
                actionData: [],
                asynchronous: true,
            });
            // And now wait for that empty import to complete, getting a reference to the // resulting collection.
            collection = await this.editorClient.awaitDataImport(DataConnectorName, projectId, CollectionName, []);
        }

	 // Add the user input data to the collection. These changes will be sent to
	 // the data-connector
        const primaryKeys = collection.patchItems({
            added: [
                {
                    [DefaultFieldNames.Project] : projectId,
                    [DefaultFieldNames.Content]: input.get(this.contentField),
                },
            ],
        });

        if (primaryKeys.length != 1) {
            throw new Error('Failed to add new card data');
        }

        return {collection, primaryKey: primaryKeys[0]};
    }
}

Finally, construct an object of this class in TodoistCardIntegration:

export class TodoistCardIntegration extends LucidCardIntegration {
    // ...

    public addCard = new TodoistTaskCreator(this.editorClient);
}

Create a new task

With both the editor-extension and data-connector running, you should be able to create new Lucid cards along with the corresponding task in Todoist.

Create Task

Examples

The source code for a basic card integration not linked to any third party can be found here. This can serve as a template for you to build a card integration.

For a more thorough example, you can look at the source code for Lucid's Asana Cards extension here. You can find the extension here if you want to try it out.