Media syncing issues and possible data loss

Damien Elmes's Avatar

Damien Elmes

27 Sep, 2016 10:25 AM

UPDATE 5:

UPDATE 4:

UPDATE 3:

  • Some accounts could not be fully restored automatically.
  • There is a good chance we can recover your data if you can provide us with the log files Anki creates.
  • Please see this page to check on the status of your account, and to see how to provide the log files.

UPDATE 2:

  • Media syncing has been restored for the majority of affected accounts
  • Restored accounts will not sync deleted files for now - please see this article for more info.
  • We're still working on restoring data for some accounts
  • If you've recovered data from one of your devices, you can manually restore syncing - please see this article for more info.

UPDATE 1:

  • The majority of the lost files have now been restored, with 2/3rds of affected users no longer missing a single file.
  • It will take me up to another day to ensure it's safe to turn back on media syncing and to finish restoring what I can.
  • For those of you who've been able to restore data from another device, I understand you're eager to start syncing again, but I do need to keep syncing of the affected accounts offline for now until I can be sure no further data will be lost.

Thank you all for your patience, and I'll post back as soon as I have new info.

IN SHORT:

  • If syncing gives you an error pointing to the announcements page, you may have lost sound or image files on one or more devices due to an AnkiWeb issue.
  • The issue only affects media files - no card text or review data has been lost.
  • To prevent media files from being deleted on any other devices you own, media syncing has been temporarily disabled for you. We hope to restore functionality in the next 24 hours.
  • You can continue to sync - your card content and study progress will still be transferred, but you will receive an error at the end of the sync because media syncing is disabled, and no media files will be transferred to other devices.
  • It is safe to study, and you will not lose any more media files than have already been lost.
  • It is safe to add new media files to your cards.
  • Automatic backups do not contain media, so restoring from an automatic backup will not help.
  • To determine what has been lost and how to restore it, please see https://anki.tenderapp.com/kb/problems/recovering-from-the-ankiweb-...

Hi all,

I discovered a serious issue with the synchronisation service today that may have resulted in sound and image files in your collection being removed erroneously if you synced Anki in the last 24 hours.

The error occurred when syncing a device that needed to download media from AnkiWeb. Instead of the file data being returned correctly, an empty file was returned in certain cases. When subsequently syncing that device again, the empty file was treated as a removal, and the files were removed from other devices when you synced them afterwards.

To clarify, if the devices you synced in the last 24 hours:
- ...did not have any media changes, then you will not be affected - ...only uploaded media to AnkiWeb, then you will not be affected - ...downloaded media from AnkiWeb, then there is a chance you are affected

I have fixed the original issue that was causing the data loss, and will be making steps to ensure that it can not happen again. To prevent already deleted files from being deleted on more devices, I have temporarily configured AnkiWeb to return an error if your account may be affected by this data loss.

I will be attempting to restore as much data as I can from backups and cached files, and will edit this post when I have more information.

This should not have happened, and I'm very sorry for any inconvenience caused. If you have any questions or need help with making backups, please don't hesitate to contact us.

Showing page 3 out of 5. View the first page

  1. 61 Posted by justindtapp on 27 Sep, 2016 11:28 PM

    justindtapp's Avatar

    Hi, when I initially noticed the affected files were missing on Monday, I simply deleted those cards and made new ones. So, I don't need a backup. Should I stop making cards with media and wait for the issue to be fixed, or is it safe to keep making them and just wait until you allow synching again-- will it accept the later version of my decks or will the server try to push old information? Thanks.

  2. 62 Posted by Chris on 27 Sep, 2016 11:33 PM

    Chris's Avatar

    I have not lost any media, but when I make new cards that include media it seems this media is not uploaded to Ankiweb, so it will not transfer to my other devices. Will this be fixed soon?

  3. Support Staff 63 Posted by Damien Elmes on 27 Sep, 2016 11:43 PM

    Damien Elmes's Avatar

    Hi everyone,

    I've updated the top post on this page to clear up some questions - please take a look at it and the article it links to.

  4. 64 Posted by Ketetha Olengue on 28 Sep, 2016 12:30 AM

    Ketetha Olengue's Avatar

    Thanks @Soren and @Thales

    Yep just trying to get my Bros cards back ha!

  5. 65 Posted by irt on 28 Sep, 2016 07:28 AM

    irt's Avatar

    When will sync be reactivated ?

    I have managed to recover the missing media from backups but not having sync working makes it difficult to use Anki across multiple devices and locations.

  6. 66 Posted by M. on 28 Sep, 2016 07:38 AM

    M.'s Avatar

    Hi again!

    I am having trouble with seeing pictures in my decks and also in shared decks that I've downloaded. Is this also because of this problem? Or this has nothing to do with it? If it's a separate problem on my programs, what should I do then to see the pictures? I'll show you what the pictures look like - i attached a screenshot.
    These icons appear both on my Ankiweb and on my Ankidroid, though not on the Desktop Anki. Could you tell me how could I solve this problem? As it is quite hard to study anatomy if I can't see the pictures...
    Also, is it normal that I can't sync yet or am I the only one?

    Thanks!

  7. 67 Posted by Сергей Югай on 28 Sep, 2016 08:17 AM

    Сергей Югай's Avatar

    >Среда, 28 сентября 2016, 10:38 +03:00 от "M." <[email blocked]>:

  8. 68 Posted by Dmytro on 28 Sep, 2016 09:02 AM

    Dmytro's Avatar

    On my iphone i have 39 missing files and 2142 - present but not used

    on mac not synced in last 24 hours - 0 missing and 2165 not used media files

    What actions should i take to not lost media will present on my mac?
    will It fix automatically when sync will be enabled?

  9. 69 Posted by Tonyo on 28 Sep, 2016 11:00 AM

    Tonyo's Avatar

    Hi!

    I lost about 10-15 files, not much and I can easily put them back but this safety thing doesn't let me sync media, is there a way to re-enable media sync on my account or I have to wait?

  10. 70 Posted by Soren Bjornstad on 28 Sep, 2016 12:04 PM

    Soren Bjornstad's Avatar

    @irt: As Damien said in the updates on his post, we're hoping sometime within the next 24 hours. Also, keep in mind that even if you get an error at the end of syncing, the previous part still completed. Your review progress will still sync right now, it's just media that won't.

    @M.: If you can't sync, that suggests you were hit by the issue, as only people who encountered it currently have media sync disabled. Please read Damien's original post in this discussion and the knowledge base article he's linked and then ask any further questions you have.

    @Tonyo: I'm afraid you'll just need to wait for a little while longer.

    @Dmytro:

    What actions should i take to not lost media will present on my mac? will It fix automatically when sync will be enabled?

    It wouldn't hurt to do a backup by closing Anki and copying your whole Documents/Anki folder onto the desktop. Then just sit tight until we get syncing safely working again, and you should be able to sync again to fix it. It's fine to review in the meantime, and you can sync your study progress even if you get an error, you just won't be able to sync any media.

  11. 71 Posted by r859ab on 28 Sep, 2016 12:13 PM

    r859ab's Avatar

    I am afraid to generate my cards again. It took like forever to make them, and now they are gone ....What a waste of time. And I trusted Anki so much. Hope I get my lost media back!!!

  12. 72 Posted by panel on 28 Sep, 2016 12:44 PM

    panel's Avatar

    Here is interesting part:

    1. I lost two media files. I can live with that. I got the error at the end of the sync, but as I understand rest of things is syncing correctly, and this error can be ignored? Is it true?

    Syncing failed:
    Traceback (most recent call last):
      File "aqt\sync.py", line 324, in run
      File "aqt\sync.py", line 385, in _sync
      File "aqt\sync.py", line 416, in _syncMedia
      File "anki\sync.py", line 736, in sync
      File "anki\sync.py", line 870, in begin
      File "anki\sync.py", line 900, in _dataOnly
    Exception: SyncError:Please see http://help.ankisrs.net/discussions/announcements

    2. Media Check showed me bunch of unused files and two missing files ones. I tried to remove error at the end of the sync, so I have created png files, named them as missing files and placed the files inside \Anki\User 1\collection.media folder. Now Media check does not show any missing files, but the sync error is still there.

    3. The lesson learned is that all media files should also be included in backup (We have plenty space on our hard drives so I believe it would be fine if you simply use a little bit more space)

  13. Support Staff 73 Posted by Damien Elmes on 28 Sep, 2016 01:13 PM

    Damien Elmes's Avatar

    Hi all,

    I've just edited the top post to add an update:

    • The majority of the lost files have now been restored, with 2/3rds of affected users no longer missing a single file.
    • It will take me up to another day to ensure it's safe to turn back on media syncing and to finish restoring what I can.
    • For those of you who've been able to restore data from another device, I understand you're eager to start syncing again, but I do need to keep syncing of the affected accounts offline for now until I can be sure no further data will be lost.

    Thank you all for your patience, and I'll post back as soon as I have new info.

  14. 74 Posted by huang.sundi on 28 Sep, 2016 01:26 PM

    huang.sundi's Avatar

    Yay!

    Will we be able to download syncs at specific timeframes? I think I may have deleted notes that had missing media after midnight and then synced before the bug was caught...

  15. 75 Posted by Joshua on 28 Sep, 2016 01:50 PM

    Joshua's Avatar

    Thanks for all your hard work Damien!

  16. 76 Posted by r859ab on 28 Sep, 2016 01:55 PM

    r859ab's Avatar

    Thanks alot. I hope to get my data back!!!!

  17. 77 Posted by Mat on 28 Sep, 2016 03:00 PM

    Mat's Avatar

    Sincronizzazione fallita:
    Traceback (most recent call last):
      File "/usr/share/anki/aqt/sync.py", line 324, in run
        self._sync()
      File "/usr/share/anki/aqt/sync.py", line 385, in _sync
        self._syncMedia()
      File "/usr/share/anki/aqt/sync.py", line 416, in _syncMedia
        ret = self.client.sync()
      File "/usr/share/anki/anki/sync.py", line 736, in sync
        ret = self.server.begin()
      File "/usr/share/anki/anki/sync.py", line 870, in begin
        "begin", StringIO(json.dumps(dict())))))
      File "/usr/share/anki/anki/sync.py", line 900, in _dataOnly
        raise Exception("SyncError:%s"%resp['err'])
    Exception: SyncError:Please see http://help.ankisrs.net/discussions/announcements

  18. 78 Posted by Mat on 28 Sep, 2016 03:02 PM

    Mat's Avatar

    doesn't work

  19. 79 Posted by M. on 28 Sep, 2016 03:21 PM

    M.'s Avatar

    @Soren Bjornstad

    Sorry, I just wanted to know if it's the problem of my old tablet that I can't see the pictures or the lost media problem. I'm a little bit afraid that maybe my tablet is just not so good that it can't show me the pictures? Because I just started syncing there yesterday and I'm not sure if I can't see the pictures because of my old and slow tablet or the lost media. That's because I asked - if it is my tablet's problem, if I even won't see the pictures after the problem has solved, then is it possible somehow to update my tablet or download some program or something to get to see my pictures? I am really sorry if this question is a waste of time to you, but I don't know much about computers and programs and would like someone's help...

  20. 80 Posted by Thales Alexandr... on 28 Sep, 2016 04:07 PM

    Thales Alexandre Ferreira Albuquerque's Avatar

    I just realized most of my missing files were the ones I copy pasted from the internet. Do you know if adding media through copy pasting makes it more prone to issues?

  21. 81 Posted by Soren Bjornstad on 28 Sep, 2016 04:43 PM

    Soren Bjornstad's Avatar

    @M.: Chances are good it's the lost media problem, so let's wait until everything is resolved with this issue, and then if you're still having trouble we can go from there.

    @Mat: Please read Damien's initial post for information on this error.

  22. 82 Posted by MeriB on 28 Sep, 2016 05:16 PM

    MeriB's Avatar

    Same as Thales, the media files that are missing are the ones I copy-pasted from the internet. Not sure if this helps.

  23. 83 Posted by ehochx72 on 28 Sep, 2016 07:04 PM

    ehochx72's Avatar

    Hey, good luck guys!

  24. 84 Posted by Pawan Solanki on 28 Sep, 2016 07:29 PM

    Pawan Solanki's Avatar

    Just wanted to check in to say I appreciate the work you guys are doing!

  25. 85 Posted by JH on 28 Sep, 2016 08:18 PM

    JH's Avatar

    Thank you for all of the work!

  26. 86 Posted by Ryan on 28 Sep, 2016 10:03 PM

    Ryan 's Avatar

    Thank you very much for all of your hard work! Your program has helped me SO MUCH at the university. I'm extremely grateful for you!!

  27. 87 Posted by AJ on 29 Sep, 2016 02:21 AM

    AJ's Avatar

    Any idea when syncing might be back up? I understand you're working hard, thanks for all that you do!

  28. Support Staff 88 Posted by Damien Elmes on 29 Sep, 2016 03:17 AM

    Damien Elmes's Avatar

    Hi all,

    Media syncing has been restored for the majority of affected accounts now. For more information, please see the update at the top of this post.

  29. 89 Posted by Nathan Smith on 29 Sep, 2016 03:51 AM

    Nathan Smith's Avatar

    Just synced and all of my media is back! Thanks!

  30. 90 Posted by huang.sundi on 29 Sep, 2016 04:09 AM

    huang.sundi's Avatar

    Synced worked (replaced deck that I had exported prior to syncing...) but then it said it failed (despite it syncing!).

    Maybe you guys are still trying to retrieve a previous cache of my synced deck?!

Comments are currently closed for this discussion. You can start a new one.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac