Media syncing issues and possible data loss

Damien Elmes's Avatar

Damien Elmes

27 Sep, 2016 10:25 AM

UPDATE 5:

UPDATE 4:

UPDATE 3:

  • Some accounts could not be fully restored automatically.
  • There is a good chance we can recover your data if you can provide us with the log files Anki creates.
  • Please see this page to check on the status of your account, and to see how to provide the log files.

UPDATE 2:

  • Media syncing has been restored for the majority of affected accounts
  • Restored accounts will not sync deleted files for now - please see this article for more info.
  • We're still working on restoring data for some accounts
  • If you've recovered data from one of your devices, you can manually restore syncing - please see this article for more info.

UPDATE 1:

  • The majority of the lost files have now been restored, with 2/3rds of affected users no longer missing a single file.
  • It will take me up to another day to ensure it's safe to turn back on media syncing and to finish restoring what I can.
  • For those of you who've been able to restore data from another device, I understand you're eager to start syncing again, but I do need to keep syncing of the affected accounts offline for now until I can be sure no further data will be lost.

Thank you all for your patience, and I'll post back as soon as I have new info.

IN SHORT:

  • If syncing gives you an error pointing to the announcements page, you may have lost sound or image files on one or more devices due to an AnkiWeb issue.
  • The issue only affects media files - no card text or review data has been lost.
  • To prevent media files from being deleted on any other devices you own, media syncing has been temporarily disabled for you. We hope to restore functionality in the next 24 hours.
  • You can continue to sync - your card content and study progress will still be transferred, but you will receive an error at the end of the sync because media syncing is disabled, and no media files will be transferred to other devices.
  • It is safe to study, and you will not lose any more media files than have already been lost.
  • It is safe to add new media files to your cards.
  • Automatic backups do not contain media, so restoring from an automatic backup will not help.
  • To determine what has been lost and how to restore it, please see https://anki.tenderapp.com/kb/problems/recovering-from-the-ankiweb-...

Hi all,

I discovered a serious issue with the synchronisation service today that may have resulted in sound and image files in your collection being removed erroneously if you synced Anki in the last 24 hours.

The error occurred when syncing a device that needed to download media from AnkiWeb. Instead of the file data being returned correctly, an empty file was returned in certain cases. When subsequently syncing that device again, the empty file was treated as a removal, and the files were removed from other devices when you synced them afterwards.

To clarify, if the devices you synced in the last 24 hours:
- ...did not have any media changes, then you will not be affected - ...only uploaded media to AnkiWeb, then you will not be affected - ...downloaded media from AnkiWeb, then there is a chance you are affected

I have fixed the original issue that was causing the data loss, and will be making steps to ensure that it can not happen again. To prevent already deleted files from being deleted on more devices, I have temporarily configured AnkiWeb to return an error if your account may be affected by this data loss.

I will be attempting to restore as much data as I can from backups and cached files, and will edit this post when I have more information.

This should not have happened, and I'm very sorry for any inconvenience caused. If you have any questions or need help with making backups, please don't hesitate to contact us.

Showing page 4 out of 5. View the first page

  1. Support Staff 91 Posted by Damien Elmes on 29 Sep, 2016 04:17 AM

    Damien Elmes's Avatar

    Yes, your account has not been restored yet. Please see https://anki.tenderapp.com/kb/problems/recovering-from-the-ankiweb-... to check on the status of your account.

  2. 92 Posted by huang.sundi on 29 Sep, 2016 04:20 AM

    huang.sundi's Avatar

    Oh sweet, thanks!

  3. Support Staff 93 Posted by Damien Elmes on 29 Sep, 2016 06:55 AM

    Damien Elmes's Avatar

    For people whose data has not been recovered yet, could you please send me your collection.log/debug.log file? Please create a new support ticket on this site and attach the file, making sure to mark the post as private so others can not see your data.

    On a computer you can copy this file from Documents/Anki/User 1/collection.log to somewhere else like your desktop. On AnkiMobile, please see http://ankisrs.net/docs/am-manual.html#debug-log

    If you have logs from multiple devices, please attach them all.

  4. 94 Posted by J on 29 Sep, 2016 07:44 AM

    J's Avatar

    Thanks for the fix!

    This issue has really shown me how much I rely on your free software. How can I donate? I don't have any Apple devices.

  5. 95 Posted by Marco Aurélio on 29 Sep, 2016 07:51 AM

    Marco Aurélio's Avatar

    All working fine just now.

    Congratulations guys, for the hard and good work.

    I pay for the important apps I use, so, if you wanna a reason, among many, I would suggest a companion program, or new anki's feature, that could allow me to point to google drive so I could make backups from the whole anki's directory automatically and scheduled.

    I can't afford loose the work I'm doing with the excelent anki program.

    Once more: tks a lot!!!

  6. 96 Posted by justindtapp on 29 Sep, 2016 09:48 AM

    justindtapp's Avatar

    To be certain, now that I have typed "I understand" on deletions, if I sync Anki-- having added many new cards since syncing was disabled and deleted & manually recreated all the affected ones where media was lost-- my decks will not be reset to a previous version, correct? I won't lose anything I've done in the last two days? Thanks.

  7. Support Staff 97 Posted by Damien Elmes on 29 Sep, 2016 10:28 AM

    Damien Elmes's Avatar

    I'm afraid we can't accept donations, but thank you for the thought :-)

    justin: Provided you followed the instructions on that page, you will not lose any data:

    Please visit the preferences screen on each device you use Anki  
    on, and select the sync/network settings, then choose  
    deauthorize, which will log you out and tell Anki to forget any  
    deletions that are waiting to sync. You can then sync again, and  
    after entering your username and password the sync should  
    complete successfully.
    
    Once you have verified that all devices are syncing successfully,  
    you can enable deletion syncing again.
    
    Please do not enable deletions until you have verified all devices  
    are syncing successfully. If you do not deauthorize all of your  
    devices, media files may be lost.
    

    If you're unsure, I'd recommend exporting your entire collection with scheduling data and media, so you can restore from it if you forgot to deauthorize a device. That's why it's best to make sure all devices are syncing prior to enabling deletions, so that you can be sure no devices have pending deletions.

  8. 98 Posted by huang.sundi on 29 Sep, 2016 10:31 AM

    huang.sundi's Avatar

    I'm having trouble submitting a ticket. Get the "We're sorry, but something went wrong"

    Is there an email I can send my collection log to instead?

  9. 99 Posted by r859ab on 29 Sep, 2016 10:31 AM

    r859ab's Avatar

    Thank you so much for your hard work. Someone here suggested a google drive integration to media. I found this suggestion really good. hope the admin thinks about it too.

  10. Support Staff 100 Posted by Damien Elmes on 29 Sep, 2016 10:33 AM

    Damien Elmes's Avatar

    huang: sigh, that looks to be caused by a recent change our support site service made. Does it work if you zip up the file(s) first? If not, another option would be to put the files on Dropbox/Google Drive/etc, get a sharing link, and then attach that in a private ticket.

  11. 101 Posted by huang.sundi on 29 Sep, 2016 10:39 AM

    huang.sundi's Avatar

    Yep, zip file works :)

    Thanks!

  12. 102 Posted by ehochx72 on 29 Sep, 2016 11:00 AM

    ehochx72's Avatar

    I think the sync works again for my account after "entering I understand".

    Many many thanks to you guys!

    Ehm, why the hell you can't accept donations? I would also like to show my thankfulness!

    Thanks!
    Daniel

  13. 103 Posted by M. on 29 Sep, 2016 11:26 AM

    M.'s Avatar

    Thank you so-so much!

  14. 104 Posted by Alex on 29 Sep, 2016 11:43 AM

    Alex's Avatar

    Thanks for the response to this issue. I would also be willing to pay for some automatic backing up of the media files.

  15. Support Staff 105 Posted by Damien Elmes on 29 Sep, 2016 01:11 PM

    Damien Elmes's Avatar

    Hi all,

    The majority of accounts have been fully restored now, but I've reached the limits of what I can recover based on the AnkiWeb data alone. There is a good chance that most of the remaining data can be recovered with the help of the log files that Anki keeps. Please see https://anki.tenderapp.com/kb/problems/recovering-from-the-ankiweb-... for more info.

  16. 106 Posted by Andrew on 29 Sep, 2016 01:53 PM

    Andrew's Avatar

    I couldn't find the option to "deauthorize" on Ankidroid. Where exactly is this setting? Looked everywhere in the preferences menu and didn't see anything like it.

    This is regarding these instructions:
    "Please visit the preferences screen on each device you use Anki
    on, and select the sync/network settings, then choose
    deauthorize, which will log you out and tell Anki to forget any
    deletions that are waiting to sync. You can then sync again, and
    after entering your username and password the sync should
    complete successfully."

    Also... out of frustration and a desire to be able to use my cards again, I ended up re-enabling syncing and deletions and tried to do a full one way sync from ankiweb to my Ankidroid device (the "Check Media" setting told me there were only 7 deletions, so I was just going to accept those deletions and move on). I saw though that it started trying to upload like 350 media changes so I quickly terminated the program in case it was trying to do more deletions. I'm not sure if even more stuff got deleted, but if so, would this be something that could be recovered?

  17. 107 Posted by Cure Dolly on 29 Sep, 2016 02:57 PM

    Cure Dolly's Avatar

    I'd just like to add my thanks for the wonderful work you do. Thank you for fixing this problem and for everything else.

  18. 108 Posted by Minh Nguyen on 29 Sep, 2016 03:10 PM

    Minh Nguyen's Avatar

    Thank you so much for fixing this issue. :)

  19. 109 Posted by Aurélien on 29 Sep, 2016 04:09 PM

    Aurélien's Avatar

    Thank you so much for your efficiency !

  20. 110 Posted by r859ab on 29 Sep, 2016 04:11 PM

    r859ab's Avatar

    Although the link characterized me to be in " Deletions Disabled", yet I am unable to receive some of my recent media. I was able to recover an old collection through a cloud, but recent is still missing. I tried to create a support ticket but it shows an error. About 5000 files are mising from PC. please give me some email address where I can get support

  21. 111 Posted by Saba sheikh bah... on 29 Sep, 2016 04:42 PM

    Saba sheikh bahaee's Avatar

    Hi
    thank u for ur hard work. But I cant recover my files yet. Would plz tell me what is the private ticket. I didnt get where should I upload my collection media

  22. 112 Posted by Serg on 29 Sep, 2016 05:26 PM

    Serg's Avatar

    Dear programmers, thank you very much!
    I have restored all my data. I love your program and updates. )

  23. 113 Posted by doctor.yanxiaof... on 29 Sep, 2016 06:32 PM

    doctor.yanxiaofeng's Avatar

    Dear Damien,

    I send a private discussion try to get help with my missing media problem this morning, but until now nobody responded! could you please help ASAP, I really need ANKI help me my coming exams! really appeciate your help in advance!

    xiaofeng yan

  24. 114 Posted by Oliver on 29 Sep, 2016 07:54 PM

    Oliver's Avatar

    This comment was split into a new discussion: Corrupt database

    Hi,

    My sync problems starts only today, first it was only with the sync Mac to AnkiWeb
    Then the 'check database' didn't work either.

    I don't use AnkiWeb or iOS device, i only upload (sync) anki mac to Ankiweb.

    It seems that i didn't loose any file at all.
    But keep getting this message when I try to sync (check database) on my MacBook :

    - see attach file

    So i tried to disconnect the automatic sync on my MacBook (as explain in older post)
    And that didn't resolve the problem at all !
    Now i can't login and i get this :

    "Your collection file appears to be corrupt. This can happen when the file is copied or moved while Anki is open, or when the collection is stored on a network or cloud drive. Please see the manual for information on how to restore from an automatic backup"

    I checked "the manual" but didn't get a clue at my answer : is it only a corrupt database or something else ?

    What can i do ?

    1. I tried to restore a backup (with another profile) but i loose all « media »
        Can i get them back ? How ?

       I have my own backups but the last one is one week old.

    2. Or can i export my Ankiweb collection to a new profile on Anki mac, and get the media ?

    I have tried everything i can.
    If some one can help me, it would be great!

  25. Support Staff 115 Posted by Damien Elmes on 30 Sep, 2016 01:53 AM

    Damien Elmes's Avatar

    Andrew: deauthorize is "logout" on AnkiDroid - I'll update the docs. You must make sure to deauthorize/logout prior to syncing your devices, or you may end up deleting the media that has been recovered.

  26. Support Staff 116 Posted by Damien Elmes on 30 Sep, 2016 01:59 AM

    Damien Elmes's Avatar

    Saba: please create a post on this site - when you create a new post, you have an option of marking it private.

    Everyone: we've been able to recover almost all of the media for users who've sent us their logs. If you're still missing media, please send your logs as mentioned in https://anki.tenderapp.com/kb/problems/recovering-from-the-ankiweb-...

    It may take us a while to process them, so please hold on - we will get to you.

  27. 117 Posted by RK on 30 Sep, 2016 02:51 AM

    RK's Avatar

    The entire media sync is aborted when Anki is trying to sync deleted media because of this prevention of media loss thing... So I cannot upload any media that follow deletion.

  28. Support Staff 118 Posted by Damien Elmes on 30 Sep, 2016 04:07 AM

    Damien Elmes's Avatar

    Please follow the steps on https://anki.tenderapp.com/kb/problems/recovering-from-the-ankiweb-... to enable deletion syncing again.

  29. 119 Posted by azat_ on 30 Sep, 2016 04:48 AM

    azat_'s Avatar

    This comment was split into a new discussion: Pictures are not syncing

    hello!
    I am trying sync new card from laptop version to ankiweb, but cards with pictures are not syncing.Only text without picture
    help

  30. 120 Posted by irt on 30 Sep, 2016 06:25 AM

    irt's Avatar

    Mr Elmes, thank you for:
    1. Communicating regularly and clearly with us during this episode.
    2. Making extensive ( and largely successful ) attempts to recover the affected user data.
    3. For your fine product.

    I am most impressed and will certainly encourage those with IOS hardware to buy your product.

Comments are currently closed for this discussion. You can start a new one.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac