Xero Line Item Sync and Unbilled Activity Link Bug

Update Dec 12:

Our team has been able to overcome the challenge with the activity links being deleted by changes coming back from Xero (in some cases after a few seconds) which was making it hard to restore activity links via our daily backup systems. As a result, line items have been repaired in production for a small number of clients and we're validating with these clients that the resolution is as desired. Pending the results of this validation/confirmation we're aiming to have the remaining activities re-linked across all client accounts by very early next week.

Update Dec 11:

Our team has now restored hundreds of line item links as a result of the Xero sync bug detailed below. One challenge that our engineering team still needs to overcome is that the time between creating the Activity links successfully and having the Xero update break them was as low as a couple of minutes, making it difficult to restore the data using the backup recovery process. Fortunately, at 1am this morning their time our senior engineers have found a way to re-institute the links via data artefacts, and hope to be able to progress further re-linking of activities on line items through the next 48 hours.

Original Post Dec 5:

On November 5, 2019, our team released an update to our Xero integration which allowed our users to edit their invoices in Xero and have the changes they made to line items (including descriptions, amounts and tax codes) synchronized automatically back into Accelo. Unfortunately, this improvement made it difficult to detect a bug which has resulted in the links between Activities (work logs) in Accelo and invoice line items to be disconnected.

The cause of this error is related to how Xero and Accelo round their quantities and amounts. Accelo provides support for five decimal places, whereas Xero only provides support for four. The consequence of this was that when we brought backline items from Xero into Accelo, we noticed that they had changed. We then performed an update on the Accelo side which resulted in us removing the linkage between activities and the line items. This has affected any invoice modified in Accelo or Xero since November 5th, and we're working on restoring these links from data backups as soon as we can. 

The consequence of this link being removed is two-fold:

1. For users who choose to provide the activity detail (work logs) in their invoices to clients, the PDF invoice would lose this work detail (since no work is linked) if they regenerated the PDF invoice.
2. For users who choose to provide the activity detail (work logs) in their invoices to clients, any invoice created before the data restore is completed will include/bring in activities that were unlinked from a previous invoice (e.g., a December invoice could include work completed in October that was invoiced in November).

Fixing these links and ensuring our clients can trust the data in Accelo is our highest priority. Our senior engineers have been working non-stop on this issue since it was uncovered (including through weekends and until well past midnight on many occasions) and we're confident we will be able to recreate these links for our users with only one exception: if a subsequent invoice is created which includes activities that were previously linked, we will not be changing the activity to point to the old invoice (since an activity can only be "invoiced for" once). 

As a result, we'd encourage clients who use the activity detail or uninvoiced features in Accelo to hold off on invoicing for their work done until we've been able to complete the restore. We don't have a firm date for when this will be, but we're anticipating it will be in the next one to two weeks. We'll continue to update this blog post with progress.

For any clients who have further questions about this, please email [email protected].