A bag of possibly unfair coins

I have a question. Given a bag with a large number (perhaps 1000) of possibly unfair coins (as in, the odds of heads vs tails might not be 50%, and different coins might have different degrees of unfairness) how could one most efficiently determine which coins (if any) were unfair or not unfair by flipping the coins and analyzing the results? Since nothing in life is ever certain, one will have to make some decisions about confidence of results (both positive and negative) and degrees of unfairness that define "fair vs unfair", so an additional question is, which decisions need to be made and how might these affect the method used for identifying which coins are unfair? The unfair coins do not need to be equally unfair, and if they are unfair I don't care how unfair they are, especially if this makes the method more efficient.

I'm not actually doing this, by the way.

Let's aim for a degree of confidence of 95%, that's always a good target.

I'm going to assume that we can't flip more than one coin at a time. If you can flip more than one at a time this gets messy. I'm sure some really clever statistician could figure out how to handle the confluence of randomness, but not me, not for free.

So we're taking each coin out of the bag, and flipping it some # of times, I say 10 to start. If you flip a fair coin 10 times, 97.9% of the time the results will be between 2 and 8 heads. Any coin that has between 2 and 8 goes in the (potentially) fair pile, sorted by the number of heads (so we can use this figure in the next step), any with more or less goes in the unfair pile.

Set aside the unfair pile, and then start flipping 'fair' coins some more, adding up the current results to old results for each coin. Maybe you go up in rounds of 10. So you've flipped these coins 20 times now. Any coin that has between 6 and 14 heads goes in the fair pile (95.8% of fair coins will), any with more or less goes in the unfair pile.

Keep doing this until you get to some larger flipping target, like 100 flips per coin. Then stop, satisfied that most coins in the unfair pile are unfair, and most coins in the fair pile are fair. With my system, any left in the fair pile have been flipped 100 times, and you can choose you final degree of confidence here. For example, 94.3% of fair coins will have between 41 and 59 heads.

Expected number of flips required depends greatly on the number of unfair coins.

If you had 1000 coins, wouldn't you need to start with a confidence level of >99.9% to avoid a serious number of false positives (or negatives)? At least if you are treating each coin individually. And no, I was not expecting that you would have to flip the coins one at a time. And yes, the confluence of randomness makes me very confused, which is mainly why I am asking. So, it is somewhat promising to me (and my potential level of understanding probability) that this may not be a trivial problem.

Would there be anything gained by *not* treating all of the coin flips of a particular coin together? For example, if you determined at a 95% confidence level that a coin was unfair, would it be somehow advantageous to run a separate trial of that coin starting from scratch? If so, how would you treat the results?

Is there any sort of trick that could be done by, for example, dividing the set of coins into two groups and looking at the results of each group to see if they acted as a set of fair coins together, and if not, then subdividing the coins further?

It seems to me very likely that there are "no shortcuts" possible in assessing the fairness of the coins, but I do not understand the situation well enough to be sure.

So I assume you want a strategy which minimizes effort (amount of flipping) while maximizing certainty of identifying coin correctly.

Given that, does flipping 1 coin 1000 times (and recording the results) require the same effort as flipping 1000 coins 1 time each (and recording the results?