The importance of conditional probability (why screening is not always a good idea)
Jade Goody died from cervical cancer. This has caused some people to call for the screening age for cervical cancer to be lowered to the age of 20. A laudable goal, perhaps? Indeed, if we have tests, why don't we screen for every type of cancer?
Sadly, unless a test is very good, there will be many false positives- a simple example: Suppose 1 in a 1000 people gets cervical cancer, and lets suppose our test is 99% accurate. That is, it will miss the disease one percent of the time, and falsely claim the disease is present when it is not 1% of the time. Bayes theorem allows us to calculate the probability that someone who is declared positive for the disease actually has it. Rather than subject you to the equation, I will explain how it actually works.
Suppose I scan 1000 people. Of those, 1 of them will (on average), have the disease, so we correctly identify this with a 0.99 percent probability. There are 999 people remaining, and our test is 99% accurate, so thats 9.99 people diagnosed falsely with the disease.
So thats 0.99 who actually have the disease, and 9.99 who do not! And this example is actually generous: Generally speaking tests are MUCH worse than this, and I'm not sure the disease is even that prevalent.
Now one can increase these probabilities greatly by repeating the test, providing that we accept that our patient has the disease only if both are positive. Still, this is a lot of cost, and worry for the patient who has endured this. This is why screening tests are generally saved for those at risk to the disease. Bear in mind that we only have finite resource, and if the NHS spends a lot on screenings tests, while some people who wouldn't be picked up won't be, who knows who will suffer thanks to the massive waste in resources in checking these hundreds who do not actually have the disease.
This result is not immediately obvious, when looking at probabilities, but it is vital, and sadly not known by the majority of people
Labels: maths, rant, statistics
To battle, brain!
Apparently, I am only capable of once a month posts. As one of those listed below was called appalling by a former commenter, I clearly better increase my quantity or my quality to make up for it! And it's unlikely to be the latter, I'd better make more posts! In this spirit, I start the new year with a post only two days in.
A year engaged, and much has changed. I am now studying a PhD, living in a rather grander flat, and have developed super powers.
Now I have the joint stresses of planning for a rapidly approaching wedding, and a rapidly increasing in pressure phd... Yay? As I expected might happen, christmas was a time which involved very little work, seeing as I was busy seeing a whole bunch of people I had not seen in a very long time, a consequence of having very good friends in random geographical locations. This means, joyfully, that I get to work nice and hard over the next few days, puzzling over some extremely tough mathematical challenges. While I enjoyed the course I went on, and felt I learnt many things, apparently I did not pick up enough to slay the questions set me with my mind powers. Hopefully this will change tomorrow morning, when I begin again afresh.
An issue when you get stuck on a question is that often there isn't anywhere to go- if you cannot think of an avenue of approach which might yield a solution, it's quite difficult to find one. There are extremely large differences between doing and understanding a question, and it's often the latter that is the hardest, which was what I struggled with today. Still, I shall persevere, and possibly even succeed, given luck and searching on google for help....
Labels: maths, my life, phd
Phd time!
I have started my phd, an exciting time, I'm sure you shall admit. I have managed to miss one tutorial I was meant to be giving already, thanks to stupidity on my part, and have done a tremendous amount of marking already. I will get paid for these things of course, which is nice.
Marking undergraduate papers finds interesting results: students who believe that cubic equations can have no roots (it makes sense if you've done any mathematics), and one who wrote a huge amount of utterly incomprehensible and unexplained maths that possibly answered the questions set, but it was not entirely clear.
Now all I have do do is a literature search to discover some techniques to find optimal designs for differential equations, and also attend two seminar/training things on monday and wednesday respectively. So easy then....
Labels: maths, my life, phd
Maths-statistics time!
Ok, enough with infinity, lets move onto something conceptually much easier. Statistics. You will, of course, remember the phrase "lies, damn lies, and statistics", which I'm pretty sure has been attributed to more than one person in it's time... Statistics don't really lie, but they can certainly be misinterpreted for someones gain. Statistics can sound impressive out of context, and without any kind of background, a statistic is fairly meaningless.
Generally, when someone says something like "57% of people hate you", they mean in a survey 57% hated you. It is very rare to talk about census data in statistics, although it of course occurs, and thats when you get your best results, because, well, you are talking about the whole population. The lies tend to occur on the lower level, but manipulation is everywhere.
The problem about statistics is that the results are not always obvious- they are sometimes counter intuitive. A nice example is tests. Lets suppose I have a great medical test that is correct 99% of the time. Awesome, you might think, lets use it!
That might not be such a good idea....
Suppose only 1 in 1000 people suffered from the illness. Then when I test the population if Britain (say) then there will be 60,000 with it- the test will catch 59,400. However there will be 59,940,000 people without it, and the test will think 599,400 of those have the disease! This is ridiculous, we have 10 times as many people who do not have the disease as do. This is why many tests that seem to be excellent are not used on a wide level, simply because you would end up scaring many many more people than you would save.
This is also worth baring in mind when you think about how strict the law should be. I suspect giving the police 99% accuracy is somewhat generous, but from this you can see that they would end up being wrong most of the time- this is partially why our legal system has to be so exacting, simply because we are not populated by criminals.
Labels: maths
Maths part 6 the nature of infinity
So i'm here. Finally. Remember all our components so far? We have our types of numbers, each set bigger than the other, remembering that a set is just a collection of objects, in this case numbers, and functions, which given a value from one set, give you a value from the other set.
And we now know functions can be "bijective". The actual meaning is not important now- although it's nice if you understand it. All you need to know that if there exists an injection from A to B then there are at least as many objects in B as there are in A- so the number of objects in B is greater or equal to that in A, and with a surjection, the opposite is true- the number of objects in A is greater or equal to that in B. So if we have both, the number in both is equal.
The great thing about the way we have set up our definition is that we don't need to go through assigning a number to each element in a set- we just have to show it's possible. So it's actually possible to apply these definitions to infinite sets.
A simple example, the natural numbers (1,2,3,4....) to the even numbers (2,4,6,8,....) Now f(x)=2x will easily give us a bijection here- we can assign a natural number to every even number. So while both sets have an infinite amount of objects, we can see that they (sort of), have the same amount. Because amounts don't necessarily mean much when we are talking about infinity, mathematicians say "cardinality" instead. But it's reasonable for you to say they have the same amount of objects.
Now this an interesting result, because one would expect the number of natural numbers to be double that of the even numbers- but is not true, at least not according to the version of counting we are using.
To give us our idea of infinity, we tend to split infinite sets into broad categories, countable and uncountable. Countable sets are ones which have a bijection to the natural numbers, that is we can assign a natural number to each member of the set. If you imagine an infinite cars driving by on the road, they will never stop, but you can certainly count them as they go.
Uncountable sets are bigger sets than the natural numbers- that is there is an injection from the natural numbers to the bigger set, but no surjection. So we cannot assign a natural number to each object. Imagine yourself plunked in the middle of an infinite desert and trying to count all the sand... there is no real order you can count them, no sensible pattern which will allow you to count them. Uncountable sets are huge, ridiculously huge in fact, which we shall talk about a little later.
Now the integers (all the whole numbers, including the negative ones) are countable- assign 1 to 0, 2 to 1, 3 to -1, 4 to 2, 5 to -2.... and so on, and you can describe each one.
Whats more, it is possible to show that even the rational numbers (all the fractions) are countable- after all they are just integers on top of natural numbers.
So where do we get uncountable numbers from? Well the real numbers are uncountable- there are a multitude of proofs for this, but all of them will involve mathematics a little too hard for here, and also I can't remember the easier ones (easier conceptually anyway. The mathematics involved is quite hard...). Bizzarely, or probably not, only the transcendental numbers (remember them- they are the extremely weird numers like pi and e) are uncountable.
Now whats odd about this is it's possible that compartively, the countable numbers take up a very tiny space in infinity, so if you were to pick out a number at random you'd always get an uncountable one- which is why, incidentally, all those physical constants are so inconviniently long..... The weird thing is that as a fourth year mathematician I know of very few examples of transcendental numbers... but they make up almost all of the numbers... crazy no?
Finally we have a notion of size in infinity, with uncountable sets sitting at the top of the roost. I have some more things to say about mathematics, and a little bit more about infinity, but for now... fare thee well.
Labels: maths
