@Bryce: Good question, and many possible responses, but here are a few.
1) Most people using Anki are using it out of their own self motivation. They are doing it, because they want to, because they want to get better, not because it is forced. So honesty/honor is mostly not an issue. Generally, self-motivated learners, learning for their own growth, tend to be honest with themselves.
2) But the second part of your question is are they accurate in assessing themselves, and that is a valid concern. Certainly there would be times that having a "computer" grade/rate you might be better, however, because Anki can be used to quiz you on so, so many different types of things, it would take a brilliant computer to be able to "test" whether you are accurate or not. Just for language alone, you might be learning/quizzing on speaking, listening, reading, writing. Each of these are very different skills for a computer to rate. And even on something like speaking, are you going for exactness of pronunciation, cadence, tone, or just the right word? And how would a computer distinguish to know these things. And over how many different languages? And that is just for a word. If it was at the sentence level, for example, how exact does the sentence need to be? If the sentence has an adverb in it, in many languages, adverbs can move to multiple places throughout the sentence and still be correct, but how would a computer know?
And that just is for something like language learning, but if you are learning something like how to play different chords in music, or recognizing Birds from images, or remembering a joke, or understanding slang, or knowing the next best chess play, etc. All of these are very different skills that require very different types of potentially advanced programming to know if you are right or not. And that doesn't even get to the point of the computer being able to determine if it was easy, normal, or hard for you. Even a measurement like "time" is quite an inaccurate way to judge difficulty. Also, if you are typing in answers, how does a machine interpret an typing mistake? Make you double space between words accidentally or forget a space between words, or hit a key/letter but it types it in twice. Something clearly just a keyboarding mistake, but a computer would naturally mark it wrong.
All that to say, since Anki is used for learning virtually anything (okay that might be a slight exaggeration), but nevertheless, it would require a brilliant, almost advanced AI-type level computer, and a deeply multi-lingual one at that, to be able to accurately judge if you are write or wrong.
Thanks for the thoughtful and comprehensive response. Totally tracking with you on most of it. I’ve seen a number of these language learning platforms and many use SRS and flashcards but still rely on the user to assess their progress. Seems to be a generally accepted methodology, even when language study is the only thing going on and there’s clearly a right and wrong answer. But, point taken on Anki and the notion that there’s so much more it can be used for.
And, while I’d agree that people motivated to learn would also be honest with their self-assessments, there is a tendency among learners to want to make progress. An automated assessment would keep you from being tempted to subconsciously tick up your results. The other concern that I didn’t mention in that post was that shared decks are sort of crap-shoots, right? As is the open-source world in general. You have to accept the idea that the people sharing the cards are legit and that you’re not committing bad information to memory. Any thoughts on this?
I loved the frank acknowledgement on the potentially mixed quality of shared decks. It is generally considered to be better for learning value if you design your own decks, but that takes a huge amount of time. I admit that while I design most of my decks, and am a huge proponent of doing so, the reality is that the convenience of getting using a shared deck is hard to pass by at times.
And my experience is that quality varies greatly. I think you are wise to be cautious. Anki has a way that users can rate and make comments on shared decks. This is helpful, but sometimes comments are sparse, especially for less popular topics. I think the only way though that the community can try to ensure quality decks to make sure that users rate poorly and leave feedback indicating why the deck is weak. I remember that there was some discussion on here a year ago, about a deck that had some clearly unneeded vulgar language in it, and users left comments accordingly to notify others who might download it.
On a small side note, when I download shared decks, as I am reviewing I try to cull out and delete notes/cards that I don't think have value to me (because I already deeply know it, or it is a close duplicate to something I already have, or it is just a low quality card).
Again, my thanks for the quick turnaround. Very much appreciate the insights on shared decks and best practices. As I move forward into the world of Anki, I’ll try to stick with the more reputable sources or to create my own materials. At the moment, it’ll be music theory and Russian. We’ll see how it goes.