Google Sheets <-> Templater: 500 errors
Incident Report for Dataclay
Resolved
This incident has been resolved.
Posted Aug 22, 2023 - 13:45 CDT
Update
We have not received any substantive updates from Google's support team since this issue occurred, but no further service interruptions have been noted. We were hoping there would be a more definitive answer and resolution, so that we could better mitigate the issue if it recurs. In the meantime, we are closing this incident.
Posted Aug 22, 2023 - 13:45 CDT
Update
We are continuing to monitor the error rates for Bot + Google Sheets. In the last day or so, error rates have dropped dramatically, so your experience should be noticeably improved.

We are still communicating with Google's support team to understand the source of the errors. While we have some more information, it's still not entirely clear why these errors started appearing regularly. We are reliant upon Google's engineering team to try to get better clarity. We will continue to update as more information becomes available.

We are also working towards a new Templater release that will include the patched panel and fixes to mitigate these types of errors in the future.
Posted Jul 05, 2023 - 19:54 CDT
Monitoring
Google Sheets errors: we are continuing to monitor traffic and any new error reports from customers. So far, the Bot customers using the patched Templater panel appear to be maintaining connection to Sheets. This still doesn't resolve the underlying issue with Google's API infrastructure responding with 500 server errors, but those errors will abate and the patched Templater panel should be able to continue Bot operations. Google's support engineering team is investigating the cause, and we will continue to update as we have more information.
Posted Jun 29, 2023 - 12:16 CDT
Identified
Current status for the Bot + Google Sheets issue: we are making a patched Templater 3 panel available to affected customers. This does not resolve the 500 errors, but it allows Templater to refresh the authorization token once the errors clear.

This is a short-term fix while we continue to work with Google to understand how to resolve the underlying 500 errors. We will likely be releasing patched versions of the full installers once we are certain that the errors are resolved.

This fix should work for ALL versions of Templater:
- Standard or floating
- Windows or macOS
- Mult-frame/multi-core (AE 2022 and newer) or Legacy (AE 2021 and earlier)

The installation method for this patch is a bit more complex than the standard Templater installer since we wanted to get something out as quickly as possible. To install the patch, first download the archive from this link based on which type of license you have:

Standard: https://assets.dataclay.com/templater-patches/Templater_3_14431-patch-release-nonfloat.zip
Floating: https://assets.dataclay.com/templater-patches/Templater_3_14431-patch-release-float.zip

Next, you'll want to navigate to the After Effects directory and go to `Support Files>Scripts>ScriptUI Panels`. Once there:
- (Optional) Rename the `Templater 3.jsxbin` to `Templater 3.jsxbin.bak`
- Copy the `Templater 3.jsxbin` from the ZIP archive directly to the `ScriptUI Panels` directory

Once that's done, you should be able to launch Templater normally via the After Effects UI or CLI and enable Templater Bot.

We will be keeping an eye on support channels over the weekend and into next week to ensure that the patch is successful. Subscribe to this incident to be alerted for any new updates.

If you have any questions, please reach out to Dataclay support.
Posted Jun 23, 2023 - 16:06 CDT
Update
We are internally testing a patched Templater panel that we hope will work around the errors and prevent the Bot from disconnecting from Google Sheets. If we believe that this resolves the issue, we hope to make the patch available for users to download in the next few hours.

We are simultaneously communicating with Google to try to understand why this occurring in the first place. We will continue to update this as we have more information.
Posted Jun 23, 2023 - 14:16 CDT
Update
We are continuing to work towards a resolution for the Google Sheets issue. At the moment, the temporary workarounds are still recommended for those who are experiencing 500 server errors and disconnections from the data source. While this appears to be an issue with Google's APIs, and not our software, we are exploring several ways to mitigate the effects. Please continue to follow updates through this channel and on our social media accounts.
Posted Jun 22, 2023 - 11:04 CDT
Investigating
We are investigating reports of an issue where Templater Bot users with Google Sheets data are encountering 500 server errors after the Bot has been running for a while. This issue only seems to affect Sheets + Bot.

Temporary workarounds:
- Close After Effects and reopen (error might recur at an undetermined interval)
- Manually set row ranges for renders instead of "Enable Bot"
- Export Sheets data to a local JSON file. With JSON, you can continue to use the Bot.

Example installable Sheets exporter: https://workspace.google.com/marketplace/app/export_sheet_data/903838927001
Or, write a custom Google Apps Script: https://thenewstack.io/how-to-convert-google-spreadsheet-to-json-formatted-text/
Posted Jun 21, 2023 - 14:46 CDT
This incident affected: Third-Party Data Sources (Google Apps Google Sheets).