I agree about your issues, particularly for words shorter than 5. I usually finish Aero 4-letter quizzes with 50-100 seconds to spare, JT with even more time to spare, and very rarely miss one, despite the fact that I'm unsure of as many as 3-5% of the words. And you're right about the cardbox as well -- I'd prefer much different settings (for how long to wait until seeing a correctly-answered word again) for 4s and 8s.
Here's what I'm trying to do now, for reviewing the 4s: run through a few hundred of them a day (there are only ~4000 so I should be done in a few days). Mark down in a separate text file (or mark as missed in quiz, saving the quiz in place each time) the words I'm unsure of -- for me, that's maybe 150-200. The definition of "unsure" I'm using is, roughly speaking, that I might pass it up in favor of a slightly worse (1-2 pts of equity) play in a tournament game due to the risk that it wouldn't be good. Then, once I have that list, I'll run through it a few times until I know all the words cold. If I were to try this again in a few months, I'd want to start from the beginning, generating a new list of "unsure" words. The problem for me is not that I haven't seen all the 4s many times over in my life; it's that words keep drifting from "sure" to "unsure" over time.
With the 5s, the percentage of words that I'm unsure of is significantly higher, and there is a significant proportion of words that I've never seen or seen only once. Also, there are more of them (~9000) and they aren't gimmes to anagram as much as the 4s are. For those I might consider using the cardbox, adding them by playability; I know that works for some. Or I could try a variant of the quiz I used for the 4s, but less monolithic: dividing the words up into segments by playability (using Quackle or the CT SuperMemo tool under Words/Study Tools) and then quizzing on the sublists of obscure stuff. I usually find that I have to take special care not to equate "missed" with "don't know," or even "wouldn't find over a board," for words shorter than 6 letters, just as I have to take care not to equate "got right" with "know." (The first word I missed in the 4s quiz was ROSY!) Mark as correct/missed both get used a lot.
NASPA (CSW) 1876, WESPA 1930
Virtuals Champion, April/May 2015