Media syncing issues and possible data loss

Damien Elmes's Avatar

Damien Elmes

27 Sep, 2016 10:25 AM

UPDATE 5:

UPDATE 4:

UPDATE 3:

  • Some accounts could not be fully restored automatically.
  • There is a good chance we can recover your data if you can provide us with the log files Anki creates.
  • Please see this page to check on the status of your account, and to see how to provide the log files.

UPDATE 2:

  • Media syncing has been restored for the majority of affected accounts
  • Restored accounts will not sync deleted files for now - please see this article for more info.
  • We're still working on restoring data for some accounts
  • If you've recovered data from one of your devices, you can manually restore syncing - please see this article for more info.

UPDATE 1:

  • The majority of the lost files have now been restored, with 2/3rds of affected users no longer missing a single file.
  • It will take me up to another day to ensure it's safe to turn back on media syncing and to finish restoring what I can.
  • For those of you who've been able to restore data from another device, I understand you're eager to start syncing again, but I do need to keep syncing of the affected accounts offline for now until I can be sure no further data will be lost.

Thank you all for your patience, and I'll post back as soon as I have new info.

IN SHORT:

  • If syncing gives you an error pointing to the announcements page, you may have lost sound or image files on one or more devices due to an AnkiWeb issue.
  • The issue only affects media files - no card text or review data has been lost.
  • To prevent media files from being deleted on any other devices you own, media syncing has been temporarily disabled for you. We hope to restore functionality in the next 24 hours.
  • You can continue to sync - your card content and study progress will still be transferred, but you will receive an error at the end of the sync because media syncing is disabled, and no media files will be transferred to other devices.
  • It is safe to study, and you will not lose any more media files than have already been lost.
  • It is safe to add new media files to your cards.
  • Automatic backups do not contain media, so restoring from an automatic backup will not help.
  • To determine what has been lost and how to restore it, please see https://anki.tenderapp.com/kb/problems/recovering-from-the-ankiweb-...

Hi all,

I discovered a serious issue with the synchronisation service today that may have resulted in sound and image files in your collection being removed erroneously if you synced Anki in the last 24 hours.

The error occurred when syncing a device that needed to download media from AnkiWeb. Instead of the file data being returned correctly, an empty file was returned in certain cases. When subsequently syncing that device again, the empty file was treated as a removal, and the files were removed from other devices when you synced them afterwards.

To clarify, if the devices you synced in the last 24 hours:
- ...did not have any media changes, then you will not be affected - ...only uploaded media to AnkiWeb, then you will not be affected - ...downloaded media from AnkiWeb, then there is a chance you are affected

I have fixed the original issue that was causing the data loss, and will be making steps to ensure that it can not happen again. To prevent already deleted files from being deleted on more devices, I have temporarily configured AnkiWeb to return an error if your account may be affected by this data loss.

I will be attempting to restore as much data as I can from backups and cached files, and will edit this post when I have more information.

This should not have happened, and I'm very sorry for any inconvenience caused. If you have any questions or need help with making backups, please don't hesitate to contact us.

Showing page 5 out of 5. View the first page

  1. Support Staff 121 Posted by Damien Elmes on 30 Sep, 2016 01:47 PM

    Damien Elmes's Avatar

    To make things easier, log files can now be uploaded directly to https://ankiweb.net/account/mediaRecoveryOptOut, and that page will also tell you how many media files are missing if any. People who've already attached files here do not need to upload them again.

    irt: that's kind of you to say - thank you.

  2. 122 Posted by Felipe on 01 Oct, 2016 12:44 AM

    Felipe's Avatar

    Hi there are no "collection.log and collection.log.old" files on my "Documents/Anki/User" and the site says that i lost only 1 file..i need my syncing

  3. Support Staff 123 Posted by Damien Elmes on 01 Oct, 2016 02:31 AM

    Damien Elmes's Avatar

    Felipe: If you don't mind losing the file, you can restore syncing at the bottom of https://ankiweb.net/account/mediaRecoveryOptOut

  4. 124 Posted by thalesalexandre on 01 Oct, 2016 05:15 PM

    thalesalexandre's Avatar

    ________________________________
    De: Nathan Smith <[email blocked]>
    Enviado: quinta-feira, 29 de setembro de 2016 00:52
    Para: [email blocked]
    Assunto: Re: Media syncing issues and possible data loss [AnkiWeb (Website) #1755]

  5. 125 Posted by Felipe on 01 Oct, 2016 05:35 PM

    Felipe's Avatar

    Thank you..worked :)

  6. 126 Posted by Sean on 01 Oct, 2016 05:59 PM

    Sean's Avatar

    Just a heads up - I recevied the email today with instructions for how to restore lost data. After following the instructions on each of my devices and syncing, etc., the files that had been lost on one devices were now lost on another device, which had not lost them in the first place. In other words, following the instructions had caused data loss on one device to lead to "spread" to the other, previously unaffected device.

    I was lucky that I had manually made a backup of my media folder on the unaffected device before following these directions, so I was able to restore the lost media, but if I had not made this backup, I would be left with less data than before I followed those directions!

    Just a heads up, as I don't want other people to lose more data in an attempt to recover their data.

  7. 127 Posted by thalesalexandre on 02 Oct, 2016 12:36 AM

    thalesalexandre's Avatar

    ________________________________
    De: Damien Elmes <[email blocked]>
    Enviado: quinta-feira, 29 de setembro de 2016 07:34
    Para: [email blocked]
    Assunto: Re: Media syncing issues and possible data loss [AnkiWeb (Website) #1755]

  8. Support Staff 128 Posted by Damien Elmes on 02 Oct, 2016 02:41 AM

    Damien Elmes's Avatar

    Hi Sean,

    Our logs seem to indicate that you re-enabled media deletions just prior to your files being deleted. As the red text on that page stresses, it's important to deauthorize and re-sync all of your devices prior to enabling media deletions in order to prevent any loss of data. I'm glad to hear you were able to recover your data however, and completely agree with you that it's a good idea to create a backup in such situations - we do suggest this on the support page.

  9. 129 Posted by AJ on 02 Oct, 2016 11:29 PM

    AJ's Avatar

    All of my media has been recovered, thanks for being so awesome at getting this bug fixed! Anki is the best!

    - sincerely,
    tired med student

  10. 130 Posted by Milo on 02 Nov, 2016 11:34 PM

    Milo's Avatar

    Sorry to revive a dead thread, but I was affected by this and Anki is no longer syncing media between devices. I do receive an error message every single time I sync through the program and when I close it.

    The error message points me to this link: http://ankiweb.net/account/mediaRecover

    For anyone else experiencing a similar problam: at first it didn't work so I went back into this thread to find the right link. Follow the instructions on the page, but before you do make sure to backup your Anki folder in case anything else goes wrong.

  11. Support Staff 131 Posted by Damien Elmes on 03 Nov, 2016 01:32 AM

    Damien Elmes's Avatar

    When you say it didn't work, do you remember what error message you were getting? Is it possible you mistyped the link?

    For anyone wondering about how this issue has been progressing, most users who needed to upload their logs have now done so, so we're now in the long tail, with only approx 2 people a day still doing so. All the recovered media has been backed up, so I will leave the recovery system in place for some time to give the remaining users a chance to do so.

    Over the last month I've been working on changes to media handling so that deleted files can be restored more easily, so an event like this should not be possible in the future.

  12. 132 Posted by Soren Bjornstad on 15 Dec, 2016 03:30 PM

    Soren Bjornstad's Avatar

    This ticket has been automatically closed due to inactivity. If you are still having issues or need to continue this discussion, please feel free to reply.

    If you are not the original poster but are experiencing a similar issue, please create a new thread and include a link to this one if appropriate.

  13. Soren Bjornstad closed this discussion on 15 Dec, 2016 03:30 PM.

Comments are currently closed for this discussion. You can start a new one.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac