Testing a d6: Let's say you've got a d6. Roll it 30 times. Keep a tally of how many times each face comes up, from 1 to 6. (Note that we expect the number of appearances from each face to be about 5; 30/6 = 5). At the end, go through the counts and subtract 5 from each, square them all, and then add them all up. For a fair die, the total at the end should be no more than 55.
Testing a d20: Alternatively, say you're looking at a d20. Roll it 100 times. Again, keep a tally of how many times each face comes up, from 1 to 20. For each of the counts, go through and subtract 5, then square them, then add them all up at the end. For a fair die, the total at the end of this process should be at most 150.
Comments: The process specified above uses the minimum number of rolls you can get away with (5 times the number of sides; more on that below). It has a significance level of 5%; that is, there's a 5% chance for a die that's actually perfectly balanced to fail this test (Type I error). There's also some chance for a crooked die to accidentally pass the test, but that probability is a sliding function of how crooked the die is (Type II error). A graph could be shown for that possibility, but I've omitted it here (usually referred to as the "power curve" for the test).
How does this work? As mentioned above, it's an application of the well-known "Pearson's chi-square test" in statistics. The process results in a random number whose probability follows a "chi-square" curve after some large number of rolls. Formally, the different faces of the die are called the potential "outcomes" of rolling the die; the count of how many times each one comes up is called the "frequency" of that outcome; and the number of times you expect to see each one is called the "expected frequency". For the "chi-square" function to be applicable, you've got to have an expected frequency of 5 or more for each possible outcome (hence the requirement for a number of rolls of at least 5 times the number of sides).
The numerical process after you're done rolling can be referred to as the (very, very common) method of finding the "sum squared error" (abbreviated SSE). The "error" is how far off each frequency was from the expected value of 5 (hence count - 5); the "squared error" is when you square that value (thus making everything positive, among other niceties); and the "sum squared error" (SSE) is when you add all of those up. If the die showed every face exactly 5 times, the SSE would be exactly zero; the more crooked it is, the more error, and hence the larger the SSE value at the end.
Normally in Pearson's procedure, you'd take each "squared error" and divide by the expected frequency, then add those, then check a table of chi-squared values to see how likely that result was (compared to your initial expectation). But since we expect every side of our dice to be equally likely, there's a simplification that I've done above. For example, for a d6 test at a 5% significance level (degrees of freedom one less than sides on the die, so df = 5), I go to a table of chi-square values and look up X^2(5, 0.05) and see 11.070. That means I would normally reject the fair-die hypothesis (i.e., the null-hypothesis H0, "there is no difference from a fair die") if X^2 > 11.070. Under Pearson's procedure this would imply the following:
Σ((O-E)^2/E) > 11.070
Σ((O-E)^2/5) > 11.070
Σ((O-E)^2)/5 > 11.070
Σ((O-E)^2) > 11.070 * 5
SSE > 55.35
(Above, the "O" stands for the actual observed frequency of each outcome, and "E" stands for the expected frequency of each side, which for us is 5 in every case). A similar simplification is done for the d20 process.
Now, if you wanted to improve the accuracy of the test you could obviously roll the die more times. You would then be able to reduce the chance for either a Type I or Type II error, but never totally avoid either possibility. (In practice, we normally keep the Type I error "significance level" fixed, and work to reduce the Type II error, thus improving the "power" of the test).
I'll end this with how you can form your own test, for any die, to whatever power level you desire. I'll assume you keep the significance level at 5%, and let E be the expected number of times each face should appear in your experiment (E = rolls/sides; must have E >= 5!). Then you will reject the H0 "balanced die hypothesis" if the SSE you get at the end is greater than X*E, where X is the table lookup for X^2(sides-1, 0.05). For a d4, X= 7.815; d6, X=11.070; d8, X=14.067; d10, X=16.919; d12, X=19.675; and for the d20, X=30.143. Have fun.
Edit: Someone in the comments asked for a look at an actual working example -- included below in a photo of my scratch paper from the last time I tested a d20.