My way of dealing with backlog

kasuha's Avatar

kasuha

20 Jul, 2013 09:56 PM

After over half a year doing 200+ cards a day I have finally snapped and failed to do any reviews for three days. That doesn't sound like much but over that time my backlog stacked up to over 500 cards and there's no way I would be able to process that in one or two days. I did not clear it in a week. And actually I don't feel like I even want to do 200 cards a day anymore so either the way Anki schedules cards for me had to change or I would give up on learning with Anki. But I think Anki is great tool for learning. It just sucks when you have backlog.

First of all, Anki using default setting fails completely when you work with backlog because it just presents you cards in due order, not looking at what the interval is. So it considers a card with one year interval as important as a card with 1 day interval if both are due the same day. Any new cards introduced when you have backlog get stuck at the end of the queue and you get to them with several days of delay when you have most probably long forgotten them.

I know that there is a recommendation that when you have a backlog you should switch between going through due cards by interval and by due date. That partially works if your backlog isn't large and if you make sure to clear it within a few days. Definitely not my case and with big and persistent backlog this method fails too because it introduces large delay on medium interval cards.

Some time ago I proposed that Anki should support also sorting cards by "urgency". Urgency is basically the ratio of how much the card is overdue to how long the card interval is. Cards with shorter intervals become more urgent much faster than cards with longer intervals. That makes sense to me because card with three days interval which is a day overdue is IMO much more urgent for review than a card with a year interval if it's overdue by the same delay. Unfortunately my proposal was not considered necessary to be added to Anki.

When the time came and I have finally created my backlog, I managed to find a way how to deal with the situation my way. It's not perfect and it's not extremely convenient as I have to empty and rebuild 10 filtered decks every day but it works for me and it works fine. So whoever is interested, I present my approximation of the urgency sorting here.

The formula to calculate the urgency is overdue/interval where "overdue" is number of days how much overdue the card will be tomorrow. This is because I want to give short interval cards (e.g. 1-day or 3-day) sufficient urgency even today.

Because (as far as I know) Anki does not support arithmetic expressions in its search strings, I have created ten filtered decks, each for certain level of card urgency - starting at U>=16 (i.e. cards overdue by 16 times their interval) and ending at U<=1/16 (i.e. cards overdue by less than 1/16th of their interval). Also the urgency calculation is approximated by step function. Expressions for the 10 filtered decks follow:

00 Urgency > 16
deck:"<source deck>" is:due -is:learn ((prop:ivl<=2 prop:due<=-15) or (prop:ivl<=4 prop:due<=-31) or (prop:ivl<=10 prop:due<=-79) or (prop:ivl<=22 prop:due<=-175))

01 Urgency > 8
deck:"<source deck>" is:due -is:learn ((prop:ivl<=2 prop:due<=-7) or (prop:ivl<=6 prop:due<=-23) or (prop:ivl<=14 prop:due<=-55) or (prop:ivl<=28 prop:due<=-111) or (prop:ivl<=58 prop:due<=-231))

02 Urgency > 4
deck:"<source deck>" is:due -is:learn ((prop:ivl<=2 prop:due<=-3) or (prop:ivl<=6 prop:due<=-11) or (prop:ivl<=14 prop:due<=-27) or (prop:ivl<=30 prop:due<=-59) or (prop:ivl<=62 prop:due<=-123) or (prop:ivl<=126 prop:due<=-251))

03 Urgency > 2
deck:"<source deck>" is:due -is:learn ((prop:ivl<=2 prop:due<=-1) or (prop:ivl<=6 prop:due<=-5) or (prop:ivl<=13 prop:due<=-13) or (prop:ivl<=29 prop:due<=-29) or (prop:ivl<=61 prop:due<=-61) or (prop:ivl<=125 prop:due<=-125) or (prop:ivl<=253 prop:due<=-253))

04 Urgency > 1
deck:"<source deck>" is:due -is:learn ((prop:ivl<=2 prop:due<=0) or (prop:ivl<=5 prop:due<=-2) or (prop:ivl<=14 prop:due<=-6) or (prop:ivl<=30 prop:due<=-14) or (prop:ivl<=62 prop:due<=-30) or (prop:ivl<=126 prop:due<=-62) or (prop:ivl<=254 prop:due<=-126) or (prop:due<=-254))

05 Urgency > 1/2
deck:"<source deck>" is:due -is:learn ((prop:ivl<=8 prop:due<=-1) or (prop:ivl<=20 prop:due<=-4) or (prop:ivl<=44 prop:due<=-10) or (prop:ivl<=92 prop:due<=-22) or (prop:ivl<=188 prop:due<=-46) or (prop:due<=-94))

06 Urgency > 1/4
deck:"<source deck>" is:due -is:learn ((prop:ivl<=4 prop:due<=0) or (prop:ivl<=24 prop:due<=-2) or (prop:ivl<=56 prop:due<=-6) or (prop:ivl<=120 prop:due<=-14) or (prop:ivl<=248 prop:due<=-30) or (prop:due<=-62))

07 Urgency > 1/8
deck:"<source deck>" is:due -is:learn ((prop:ivl<=16 prop:due<=0) or (prop:ivl<=64 prop:due<=-3) or (prop:ivl<=144 prop:due<=-8) or (prop:ivl<=304 prop:due<=-18) or (prop:due<=-38))

08 Urgency > 1/16
deck:"<source deck>" is:due -is:learn ((prop:ivl<=48 prop:due<=-1) or (prop:ivl<=160 prop:due<=-4) or (prop:ivl<=352 prop:due<=-10) or (prop:due<=-22))

09 Rest Due
deck:"<source deck>" is:due -is:learn

Other parameters of these decks are no limit on number of cards (9999) and random order. Values of these parameters are not that important, though.

Every day I empty all these decks, then rebuild them in order from 00 to 09 (you need to fill them in order because e.g. card with urgency 4 would be also selected by deck for urgency 2). I have decks organized this way:

main deck
- 00 filters
- - 00 Urgency > 16
- - ...
- - 09 Rest Due
- 01 current
- - <my decks from which the due cards are filtered>

And I study the main deck.

It works for me nicely so far, I don't feel any problems answering cards. At the start I had some cards in the Urgency > 4 category but now I usually get some cards in Urgency > 1 category only and every day I press it as low as possible, usually getting safely below Urgency = 1/4 (meaning cards left will not be overdue by more than 1/4 of their interval tomorrow).
Another effect is that using standard new card introduction, not reviewing all selected cards means I also get less new cards which is lowering the long term load and promises that I will be able to remove most of the backlog over time even if I keep my current load of about 100 cards a day.

The only problem I see with it is that it screws up slightly the "ease" parameter because as you get to reviewing cards of late, Anki stretches the interval of cards more than they deserve which may cause these intervals to be too long if you manage to remove the backlog. But I'm afraid that has to wait for Anki authors to come up with some better way of dealing with backlog built right into the Anki.

So well... just in case anyone finds this as useful as I do, it's here for you.

  1. 1 Posted by heather on 21 Jul, 2013 01:48 AM

    heather's Avatar

    This looks interesting. I may implement this for our Sept backlog - except we have a lot of decks that we review separately! I may cut it down to 4 or 5 urgency levels.... I'll see how much our backlog is.... I do like the Urgency concept.

  2. 2 Posted by frostschutz on 21 Jul, 2013 11:36 AM

    frostschutz's Avatar

    I do it in a similar way (three levels)

    first "is:due prop:due=0" (due today)
    just "is:due prop:due<0 (prop:due>-7 or prop:ivl<=14)" (due recently OR low interval)
    over "is:due prop:due<=-7 prop:ivl>14" (everything else overdue)

    If you do "first" every day (regular due), the just/over pools won't grow; so you study the regular due cards as if there wasn't any backlog, and add the others as you find the time.

  3. 3 Posted by kasuha on 21 Jul, 2013 05:45 PM

    kasuha's Avatar

    heather> As a minimum set I'd leave U>2, U>1, U>1/2 and U>1/4 (and Rest Due). If you created really large backlog you may perhaps use U>4 or U>8, too.

    frostschutz> It's definitely not even remotely similar to my method. It is similar to method I saw here recommended before with all drawbacks I mentioned above. It completely breaks apart if you don't manage to review the "due today" block every day and it stretches intervals of cards with intended interval above 7 days indefinitely. My method does not go after reviewing all "due today" cards every day. For example a card which is due today but has interval of 30 days will fall into the U>1/16 deck today so I will probably not review it - instead I will have time to review cards which are much more in need to be reviewed. But a card due today with interval 1 day will fall into the U>1 deck and I will very likely get it rather early on during my reviews.

  4. 4 Posted by frostschutz on 21 Jul, 2013 09:44 PM

    frostschutz's Avatar

    Your method has drawbacks as well, just different ones. I don't think there is a "correct" way to handle backlogs, just lots of different expectations. I'm usually through the today's due cards twice as quick as any overdue ones, so delaying them would be harmful in my case.

    If Anki offered some options in dealing with backlogs, such as urgency first, or recently-become-due first, that would be a great thing. If Anki insists on showing me lost causes (long overdue cards) first then I get frustrated so quickly I never ever make it through the backlog.

  5. 5 Posted by kasuha on 22 Jul, 2013 08:18 AM

    kasuha's Avatar

    I'm not saying my method has no drawbacks but of the two I have tried so far (yours and mine) my method feels much smoother and easier. Although I have over 600 cards backlog by now, none of them is delayed significantly (cards below 7 days interval are not delayed at all with my current progress, cards with interval 8 to 16 days are delayed by no more than 1 day etc). My success ratio has actually improved as I am no longer pressing myself to doing all cards for the day and can stop when I feel tired.
    You say cards from backlog are harder to review for you. That was exactly my case when I was using that method. With my method I am not feeling anything like that.
    I am not asking you to like or use my method, you're free to choose any method you prefer. I'm just assuring you that your method is very different from mine.

  6. 6 Posted by kasuha on 26 Jul, 2013 12:13 PM

    kasuha's Avatar

    After some more time spent beating my backlog I thought I could come with some further observations:

    The overall difficulty of reviews is getting the easier the longer I use the method. First day I cleared cards from U>4 through U>1/2, after that I never had any card with U>2 or more again and I gradually got from clearing up to categories U>1/4, and U>1/8. Today I am making deep cuts into the U>1/16 category already (it still contains too many cards to clear in one day but I believe in two or three days it will be gone as well) and most of cards in that category are rather easy to review (no wonder, they are only insignificantly delayed and most of them are long interval cards which I already know well). Of course, every day I get some U>1 cards (new cards introduced during reviews) but there is very few of them.

    Number of cards I was able to review each day corresponds to that. I did not manage to review even 100 cards the first three days (and my backlog grew noticeably over that time) but e.g. yesterday I reviewed almost 200 cards without problems in the same time and with about the same effort. There's still over 400 cards on my backlog from about 500 at the beginning, but their distribution is different and I can feel I am getting up to speed to clear it in a few days.

    It allows me to learn according to my physical state. If I feel tired, I can review less cards but as I review the most important cards first I can rest assured that no disaster is awaiting me tomorrow. This also makes me more willing to return to it tomorrow.

    It also adapts introducing new cards (I am using "stock" decks available here on site so I have plenty of new cards to be introduced into learning) - the bigger backlog I have the less new cards I get to learn. This acts as a negative feedback and means the overall difficulty adapts to my abilities.

    In general it turned out even better than I initially thought. I seriously suggest Anki authors to give it a chance and consider adding this feature (it's nothing more than "sorting according to urgency") into next version of Anki - preferably as the default method instead of current "by due date". It can be even adapted into sensible "review in advance" approach as cards with negative delay can be sorted using exactly the same criteria (after you avoid some 1/0 problems). Reviewing in advance may allow you to interleave it with introducing more new cards than your daily limit and with this you may even forget about distinguishing between due and not due cards - every day you can review just as many cards as you want and you get the more cards to learn the better you are at it.

  7. 7 Posted by heather on 26 Jul, 2013 04:02 PM

    heather's Avatar

    I just want to report that we used it for a few days to clear up a backlog (fewer urgency levels (>8, >4, >2, >1, >1/2, rest due, new) and it went GREAT. It seemed to give a whole new freshness to the study. We were seeing cards that were newer that we hadn't seen in a while. Results seemed great. I admit there could have been a placebo affect.... but I don't think so!

    I'm not sure if I'd want it the default sort order all the time - but it would sure be great to be able to switch to it whenever I start to get a backlog or when the reviews seem to be getting 'stale'....

  8. 8 Posted by Soren Bjornstad on 09 Aug, 2013 10:57 PM

    Soren Bjornstad's Avatar

    Another report from a satisfied user. :-)

    I had around 2500 due cards and cleared everything up in 412 minutes over 9 days. I've attached the review count graph, as it's interesting (at least for me) to see the percentage of young cards steadily dropping and mature cards growing as I got through the ones that were more likely to be forgotten and on to the ones that were safe.

    I made one change to the technique as listed in the original post: I added a "Due Today" deck with first priority above Urgency > 16. While the OP is correct that this is mathematically less efficient, I found it nice to know that even if I made no progress on the backlog, as long as I got through that top deck I at least wouldn't be making it worse, and the queue of cards underneath never grew, the cards that were in there from the beginning just shifted categories occasionally.

    I found it wasn't nearly as annoying to empty/rebuild all the decks as I expected; I just pressed /00, /01, and so on and alternated with 'e', then the same for 'r'. I don't have any other decks with the two-digit leading-zero number, so that helped; if you do, I might suggest making it a triple zero or putting a symbol in front of it so you can easily select the decks from / for this purpose.

    Thanks for the tip, and I'll be keeping the filters in my collection in case it ever gets this bad again (oh, who am I kidding, I'm sure it will sometime!).

  9. 9 Posted by heather on 10 Aug, 2013 07:53 PM

    heather's Avatar

    The minor change I made was I put at the top a filter for is:learning so that any that we are learning still get practiced.

  10. 10 Posted by kasuha on 11 Aug, 2013 06:02 AM

    kasuha's Avatar

    I excluded cards in learning from my decks for simple reason: I don't like what Anki does to them when I empty the deck. I get some learning cards in them anyway when I fail the review, but any learning cards originated from new cards stay in their original decks and don't get affected. When it happens that I have some learning cards since yesterday, I always try to review them first before I rebuild my decks.

    For comparison, I append my review count graph for almost a month of using this method. I am not aiming at clearing the backlog (although I believe it will happen eventually; I was getting close but then I skipped a day again, and one again yesterday, it was a hefty day), I am rather aiming at adjusting the rate of my learning to about 100 reviews a day since there's no way I can do 200+ as before. You can see that the ratio of young to mature cards is greatest after every drop-out but it then gradually balances out. It's of course also affected by the fact that I am no longer adding 20 new cards each day, it dropped to about 3-5 proportional to size of my backlog.

    I wish there was a graph showing number of new cards introduced. Wasn't there one in Anki 1?
    It would also help greatly if there was a graph showing my success ratio per day and per interval length instead of what's in the Answer Buttons summary.

  11. 11 Posted by Heather on 11 Aug, 2013 02:55 PM

    Heather's Avatar

    What happens to learning cards in a filtered deck when you empty the deck?

  12. 12 Posted by Soren Bjornstad on 11 Aug, 2013 04:22 PM

    Soren Bjornstad's Avatar

    Currently they're turned back into new cards. Obviously this isn't really what most people want, and Damien agrees, but he's stated it will take some time to change the behavior as it will be fairly complicated to make it do anything else.

  13. 13 Posted by Heather Stovold on 11 Aug, 2013 04:39 PM

    Heather Stovold's Avatar

    Ok, good to know... I'll obviously change that then.

    Sent from my Windows Phone
    ------------------------------

  14. 14 Posted by frostschutz on 11 Aug, 2013 09:16 PM

    frostschutz's Avatar

    I made a small addon (work in progress): https://github.com/frostschutz/anki-urgency

    It gives you a prop:urg to filter with. urg being (due+1.0)/ivl. So you can do a filter like "is:due prop:urg>=0.5 prop:urg<1.0" or whatever.

    It doesn't really seem to fit with the filters you posted above, though. Not sure if this is simply inaccuracy due to steps, or if the math should be different... Maybe you could check?

    Maybe it would be more user friendly if the urgency was expressed as a plain integer (instead of float) somehow, I'm not sure how to best go about that.

    I haven't found out how to do a custom "order" yet, or I would've added that in the first place, instead of a prop filter.

  15. Support Staff 15 Posted by Damien Elmes on 12 Aug, 2013 02:06 AM

    Damien Elmes's Avatar

    As this is not really a support question, it'd be great if you could continue this discussion over on the users forum:

    https://groups.google.com/forum/?fromgroups#!forum/anki-users

  16. 16 Posted by kasuha on 12 Aug, 2013 06:01 AM

    kasuha's Avatar

    I'm sorry, I did not notice there's a link to community forum hidden at the bottom of the support page. If I could propose an improvement I'd suggest to put a link to it to the Anki main page right beside the Support link.

    Is there a separate "propose a new feature" forum, too, or which of the two places is supposed to be used for that?

  17. 17 Posted by Soren Bjornstad on 12 Aug, 2013 02:35 PM

    Soren Bjornstad's Avatar

    Here is generally where you should do that, unless you want to get feedback from other users or discuss changes to your idea first.

  18. Soren Bjornstad closed this discussion on 12 Aug, 2013 02:35 PM.

Comments are currently closed for this discussion. You can start a new one.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac