Every so often, I use what I consider a basic term in conversation or online and come across complete ignorance that the term even exists on the part of my audience. I will here attempt to define ‘reporting artifact’ for future reference.
One often sees in the news a report that ‘twice as many cases of cancer have been diagnosed as last year’. The natural conclusion is that twice as many people got cancer. This is, in fact, a conclusion COMPLETELY unsupported by the data presented. One way that it may be a false conclusion (there are several ways) is due to the possibility of a reporting artifact. In this specific case, suppose ten times as many people were screened for cancer as last year. Would the doubling of diagnoses in a such a case indicate an increase in cancer? Quite the opposite – that would indicate that the cancer is one fifth the prior year’s level. That is a reporting artifact. Broadly stated, if you find something more only because you LOOKED for it a LOT more, it is a reporting artifact. The flip side is too: if you stop looking for something and stop finding it that, again, is a reporting artifact.
Criminal statistics are particularly subject to reporting artifacts. The reason is quite simple: people do not always report crimes. Certain crimes are more likely to be reported than others; homicide, for example, produces a body which will usually be found at some point. Robbery produces no such gross physical evidence. You can not PROVE that a homicide has NOT occurred (unless you can produce everyone who might have been murdered), but the fact that bodies tend to turn up eventually and that people wonder what happened to the victim means we can reasonably conclude that the fact that a murder has happened becomes part of a statistical set most of the time. Not so with robbery – even if you could demonstrate that everything is where it should be that doesn’t prove there weren’t TWO robberies; the second reversing the perpetrator and victim of the first.
The situation is aggravated by the fact that most crimes go unsolved (especially minor ones) and reporting a crime has negative consequences even for the victim. If nothing else, paperwork and lost time. This produces a negative incentive to report a minor crime; if someone steals a small amount of your money you might be better off working to make more money rather than spending the same amount of time working with the police to solve the crime. The crime might never be solved, and even if it is you might not get your money back.
Rape is often recognized as an under-reported crime despite being much more serious. Some of the same factors are in play; what has been lost cannot be recovered, even if the assailant is found the “I say/you say” dynamic frequently prevents a conviction, and even a conviction provides no guarantee that the assailant will not rape someone else in the future. Toss in various cultural stigmata against the victim and the fact that to report the event they must discuss taboo issues with strangers. Yes, modern US culture has taboos.
Attempts to compare rape statistics are thus subject to a very high likelihood of reporting artifacts.
As noted above, disease and injury are another common place where reporting artifacts occur. Just about everyone who has a heart attack shows up in statistics somewhere (or so we can reasonably assume; again, we can’t prove it all that well). Just about anyone, however, knows someone who got a papercut. Most of them do not show up in statistics (ER visits, EMS calls, etc., though some people do in fact seek advanced medical treatment for papercuts – boy do I wish I was making that up). So if we have a statistic that shows that there are more reported heart attacks than paper cuts, does that mean that heart attacks are more likely? Probably not.
If we posted a reward for reporting paper cuts we would introduce yet another reporting artifact; people being people, someone will deliberately give themselves a paper cut in order to get the reward. If we are trying to collect data on accidental papercuts we have just distorted our data set.
Even polling can’t eliminate reporting artifacts; people will lie, or forget, or misunderstand what is being asked. So if we poll people about their papercuts, some will forget how many they’ve gotten, some will deny they went to the hospital, and some will assume that we were asking about all accidental cuts, not just ones from paper. Note this is for something with minimal emotional impact for most people. An issue with political or emotional overtones will usually be worse (gunshots, rape, etc.).
The root of a reporting artifact is that what happens is different from what is observed, and what is observed is different from what is recorded.
Reporting artifacts can be identified without allowing meaningful analysis, especially if multiple artifacts may be at work. Cracks in airplane wings happen. People often, but not always, find them. When they find them they usually, but not always, report them. If a regulatory agency tells operators to go look for specific cracks in specific places they are more likely to find those cracks. Big cracks are more likely to be reported than small cracks. 1st world airlines are more likely to find cracks than 3rd world airlines. 3rd world airlines are more likely to have cracks (older airplanes) than 1st world airlines. If a database lists ten times as many cracks in 1st world airplanes is it because 3rd world airlines aren’t finding them, or are finding them but not reporting them? Can we know the actual ratio of cracks? Can we prove that the 3rd world airlines have any significant number of unreported cracks? The last is possible, though only with extreme care could such information be used to determine a ‘real’ ratio.
That is the real problem of reporting artifacts: even if you can prove they exist, it is far harder to adjust for them.
Let all those who compare statistics beware.
Sunday, March 17, 2013
Friday, March 15, 2013
Precedent
Lately, I’ve been noticing more and more appeals to precedent in arguments. Note, I’m not innocent of this myself.
The idea that precedent is definitive is, in my opinion, highly flawed. Precedent (assuming that it is, in fact, precedent and not something unrelated to the issue in question) is certainly relevant to a discussion or debate, but it should not be considered authoritative.
Precedent is what someone else, in an earlier time, in a similar but not identical situation (since no two situations can ever be TRULY identical), did. It does not mean, please note, that what they did was right or had a good outcome even then. If someone decided to round up six million Jews and kill them there’s a precedent for that. It would be a very, very bad precedent to follow.
Precedent is, however, worth noting. If you can extrapolate your future position from your current position without being informed by one or more precedents you are quite lucky. Most actions have unforeseen consequences. If you are able to look at the unforeseen consequences of similar actions to those you are contemplating in the past, then those particular unforeseen consequences won’t be unforeseen. They may still happen, of course, but at least you can try to prepare for them.
Precedent seems to be especially prevalent in the modern US judicial system, taking precedent (gotta love the English language) even over the written law. This often angers me. If nothing else, it should be noted that judicial precedents are often overturned. Surely something that is repeatedly shown to be flawed should not be relied upon?
As an engineer, of course, I use precedent every day. We call it ‘test results’ or something like that, but we operate on the basic premise that because something happened once (an object of such-and-such characteristics failed under such-and-such conditions) it can, and at some point probably will, happen again. If we have a lot of matching precedents (ten similar objects that all failed under similar conditions) we predict the future with confidence (another similar object will fail under the same conditions). Without precedent there would BE no engineering. You can’t do calculations without some basis to do them upon.
There is, however, a key difference. Engineering is quantified, and a chaotic element is hammered into the brains of up-and-coming engineers. Any good engineer allows a ‘factor of safety’ based on how good their precedents are and the consequences of failure. When precedents fail to be predictive we study them to learn why. And quantify that data and make it part of the NEW precedent.
Human lives are, at present, not subject to being quantified as individuals. The chaotic factors are too high, the variables too uncertain. Engineers can predict with very high confidence the minimum level of force and the application of it needed to destroy a piece of metal of known type. Not so human beings. Part of that is our variability – note the ‘of known type’ bit. If I don’t know whether I’m hitting aluminum or steel I have no idea how much force to apply. Without knowing the grade of steel and its characteristics (hardening? Temperature?) I likewise don’t know enough. Human beings are not so easily tabulated. Our actions are even less so.
Humans in groups are a little easier – as with any large system, the mistakes tend to cancel each other out. This is a basic principle of system engineering – you need not assume the worst case for all your material properties. Similarly you need not assume that all people are at the bottom end of the bell curve.
Again, though, there are parallels: good engineering insists on recognizing that some of your components WILL be at the bottom of spec (or even below it). We must also recognize that some humans in any large group will be at the bottom of the moral bell curve. They are our criminals.
There are, however, even more important divergences. Again, humans are not easily tabulated. If one is presented ten samples of a steel tempered to ten increasing levels of hardness, provided with data for all but the fifth and asked to predict the properties of the fifth an engineer can do this with high confidence. This is called interpolation, and is possible because one has carefully controlled all but a single variable. Multi-variable systems require many more data points, and have a strong tendency to have points of chaotic behavior. Many bridges had been built similar to the Tacoma Narrows Bridge (famously the ‘Galloping Girdy’). Only that EXACT bridge under those EXACT wind conditions was destroyed by resonance. The same bridge in another place might have lasted a hundred years. Had a dozen similar bridges been erected in the same place both ones of greater and lesser structural integrity would almost certainly have survived. A bridge of equivalent ‘strength’ but different design would have survived. Interpolation based on even very good data could easily have produced a failure like the GG (i.e. building multiple similar bridges in the same area in a range of stiffnesses, size, etc. bounding the intended design). A bridge is a complex system. So are people… except people are much worse.
Worst of all, humans, unlike any other material dealt with by engineers, are self willed. Steel does not WANT to break (or not break). A bridges does not CARE if it stands or falls. Again, I, like most engineers and non-engineers I know, routinely imply that this is not the case. We ascribe motivation to objects when they fail (or don’t fail when we expect them to), or blame it on the gods, take your pick. Humans may be aware they are being measured and their actions predicted and may choose to act on that information. Steel doesn’t know it is about to be cut, and can’t harden itself to resist (or soften itself to be ‘helpful’). Humans may choose to do either one.
And so we get back to my contempt for judicial precedent. Judicial precedent is an attempt to handle an individual by its nature – large groups are rarely involved. A single, original precedent is typically cited, rather than the body of decisions already based on that precedent. Again, note my contempt is not for the law – setting a standard is a necessary stage in gathering data. Worst of all, the huge range of variables present between any two human beings are routinely ignored in favor of a few similarities.
As yet, we cannot even truly deal with human bodies in engineering terms (most medicines, for all their extensive testing, would be considered to have unacceptably high failure rates in most engineering disciplines). Human brains are, from the limited evidence available, much worse.
I say, leave precedent to the engineers. The justice system can have it only after the medical system gets its error rate down.
The idea that precedent is definitive is, in my opinion, highly flawed. Precedent (assuming that it is, in fact, precedent and not something unrelated to the issue in question) is certainly relevant to a discussion or debate, but it should not be considered authoritative.
Precedent is what someone else, in an earlier time, in a similar but not identical situation (since no two situations can ever be TRULY identical), did. It does not mean, please note, that what they did was right or had a good outcome even then. If someone decided to round up six million Jews and kill them there’s a precedent for that. It would be a very, very bad precedent to follow.
Precedent is, however, worth noting. If you can extrapolate your future position from your current position without being informed by one or more precedents you are quite lucky. Most actions have unforeseen consequences. If you are able to look at the unforeseen consequences of similar actions to those you are contemplating in the past, then those particular unforeseen consequences won’t be unforeseen. They may still happen, of course, but at least you can try to prepare for them.
Precedent seems to be especially prevalent in the modern US judicial system, taking precedent (gotta love the English language) even over the written law. This often angers me. If nothing else, it should be noted that judicial precedents are often overturned. Surely something that is repeatedly shown to be flawed should not be relied upon?
As an engineer, of course, I use precedent every day. We call it ‘test results’ or something like that, but we operate on the basic premise that because something happened once (an object of such-and-such characteristics failed under such-and-such conditions) it can, and at some point probably will, happen again. If we have a lot of matching precedents (ten similar objects that all failed under similar conditions) we predict the future with confidence (another similar object will fail under the same conditions). Without precedent there would BE no engineering. You can’t do calculations without some basis to do them upon.
There is, however, a key difference. Engineering is quantified, and a chaotic element is hammered into the brains of up-and-coming engineers. Any good engineer allows a ‘factor of safety’ based on how good their precedents are and the consequences of failure. When precedents fail to be predictive we study them to learn why. And quantify that data and make it part of the NEW precedent.
Human lives are, at present, not subject to being quantified as individuals. The chaotic factors are too high, the variables too uncertain. Engineers can predict with very high confidence the minimum level of force and the application of it needed to destroy a piece of metal of known type. Not so human beings. Part of that is our variability – note the ‘of known type’ bit. If I don’t know whether I’m hitting aluminum or steel I have no idea how much force to apply. Without knowing the grade of steel and its characteristics (hardening? Temperature?) I likewise don’t know enough. Human beings are not so easily tabulated. Our actions are even less so.
Humans in groups are a little easier – as with any large system, the mistakes tend to cancel each other out. This is a basic principle of system engineering – you need not assume the worst case for all your material properties. Similarly you need not assume that all people are at the bottom end of the bell curve.
Again, though, there are parallels: good engineering insists on recognizing that some of your components WILL be at the bottom of spec (or even below it). We must also recognize that some humans in any large group will be at the bottom of the moral bell curve. They are our criminals.
There are, however, even more important divergences. Again, humans are not easily tabulated. If one is presented ten samples of a steel tempered to ten increasing levels of hardness, provided with data for all but the fifth and asked to predict the properties of the fifth an engineer can do this with high confidence. This is called interpolation, and is possible because one has carefully controlled all but a single variable. Multi-variable systems require many more data points, and have a strong tendency to have points of chaotic behavior. Many bridges had been built similar to the Tacoma Narrows Bridge (famously the ‘Galloping Girdy’). Only that EXACT bridge under those EXACT wind conditions was destroyed by resonance. The same bridge in another place might have lasted a hundred years. Had a dozen similar bridges been erected in the same place both ones of greater and lesser structural integrity would almost certainly have survived. A bridge of equivalent ‘strength’ but different design would have survived. Interpolation based on even very good data could easily have produced a failure like the GG (i.e. building multiple similar bridges in the same area in a range of stiffnesses, size, etc. bounding the intended design). A bridge is a complex system. So are people… except people are much worse.
Worst of all, humans, unlike any other material dealt with by engineers, are self willed. Steel does not WANT to break (or not break). A bridges does not CARE if it stands or falls. Again, I, like most engineers and non-engineers I know, routinely imply that this is not the case. We ascribe motivation to objects when they fail (or don’t fail when we expect them to), or blame it on the gods, take your pick. Humans may be aware they are being measured and their actions predicted and may choose to act on that information. Steel doesn’t know it is about to be cut, and can’t harden itself to resist (or soften itself to be ‘helpful’). Humans may choose to do either one.
And so we get back to my contempt for judicial precedent. Judicial precedent is an attempt to handle an individual by its nature – large groups are rarely involved. A single, original precedent is typically cited, rather than the body of decisions already based on that precedent. Again, note my contempt is not for the law – setting a standard is a necessary stage in gathering data. Worst of all, the huge range of variables present between any two human beings are routinely ignored in favor of a few similarities.
As yet, we cannot even truly deal with human bodies in engineering terms (most medicines, for all their extensive testing, would be considered to have unacceptably high failure rates in most engineering disciplines). Human brains are, from the limited evidence available, much worse.
I say, leave precedent to the engineers. The justice system can have it only after the medical system gets its error rate down.
Saturday, February 23, 2013
SET
I have for many years been a member of SET – the Society for the Elimination of TLAs (Three Letter Acronyms).
SET is an advocate for TAE – Total Acronym Eradication.
In addition to being a founding member of SET, I am also a SVP (Senior Vice President).
I helped found SET after realizing that I had bought something at EMS (Eastern Mountain Sports) for use in EMS (Emergency Medical Services).
This, of course, called to mind one of my favorite lines from the movie “Good Morning Vietnam”: “Since the VP is such a VIP should we keep the PC on the QT?”
While I was at CMU (Carnegie Mellon University, not to be confused with Central Michigan University), or more accurately while I was at CIT (Carnegie Institute of Technology, one of the colleges that make up CMU) one of the senior SCS (School of Computer Science, another of the colleges in CMU) professors was asked what he thought the fundamental limits of computer science would be. He thought for a moment, then answered “there are only 17,576 possible three letter acronyms.” It is my opinion that he was at least partially serious.
Boeing, of course, is extremely fond of acronyms. We use AP for Air Plane, or A/C for Air Craft. L/N for Line Number. AOG for Aircraft On Ground (which means it has been grounded, and thus an urgent situation, not that it is just unloading at an airport). Every airline has a three-letter customer code. Even airlines which only have two words in their name. Boeing itself, of course, has BCA (Boeing Commercial Aircraft), BDS (Boeing Defense and Space), and EOT (Engineering Operations and Technology) divisions.
I half expect that at some seniority level (say, 25 years with the company) I will be taught the secret handshake and told that Boeing is actually an acronym – Bringing Overall Engineering Into a New Globe or something. Perhaps Big Overbearing Executive Idiots... or perhaps we shouldn't go there.
SET is an advocate for TAE – Total Acronym Eradication.
In addition to being a founding member of SET, I am also a SVP (Senior Vice President).
I helped found SET after realizing that I had bought something at EMS (Eastern Mountain Sports) for use in EMS (Emergency Medical Services).
This, of course, called to mind one of my favorite lines from the movie “Good Morning Vietnam”: “Since the VP is such a VIP should we keep the PC on the QT?”
While I was at CMU (Carnegie Mellon University, not to be confused with Central Michigan University), or more accurately while I was at CIT (Carnegie Institute of Technology, one of the colleges that make up CMU) one of the senior SCS (School of Computer Science, another of the colleges in CMU) professors was asked what he thought the fundamental limits of computer science would be. He thought for a moment, then answered “there are only 17,576 possible three letter acronyms.” It is my opinion that he was at least partially serious.
Boeing, of course, is extremely fond of acronyms. We use AP for Air Plane, or A/C for Air Craft. L/N for Line Number. AOG for Aircraft On Ground (which means it has been grounded, and thus an urgent situation, not that it is just unloading at an airport). Every airline has a three-letter customer code. Even airlines which only have two words in their name. Boeing itself, of course, has BCA (Boeing Commercial Aircraft), BDS (Boeing Defense and Space), and EOT (Engineering Operations and Technology) divisions.
I half expect that at some seniority level (say, 25 years with the company) I will be taught the secret handshake and told that Boeing is actually an acronym – Bringing Overall Engineering Into a New Globe or something. Perhaps Big Overbearing Executive Idiots... or perhaps we shouldn't go there.
Saturday, February 9, 2013
Happiness
A reporter on NPR today noted that the object of the things government does is to make people happy.
While I can’t refute that many politicians and bureaucrats might claim this is the case, I don’t believe that is, nor should it be, the case.
The purpose of government is not to make us happy. The purpose is to make us safer, and to do things that we could not effectively accomplish as individuals. The actual reason governments do things is almost always to help a politician get re-elected or to allow them to do something that will make THEM happy. Of course, it probably helps your chances of re-election if your constituents are happy, but big campaign contributions help more.
There’s a quote attributed to Henry Ford, along the lines of “if I had asked people what they wanted, they would have told me they wanted faster horses.” The goal of government should not be to give people what they WANT. It should be to give them what they NEED – even if they don’t know they need it. What the people, inevitably, WANT is bread and circuses – for free, of course. What they need are standard weights and measures, clean water, armed forces, etc. Sure, if you ask people “do you want those” they’ll usually say yes – but if you just ask them what they want they’re unlikely to make the top ten.
Yet another quote is often told as a joke about a French political activist who sees a crowd running by. He runs after them, saying “I must see where My People are going so that I can lead them!” This is, in fact, a good way to get elected. It is NOT leadership. It is NOT what governments should do. What a leader would do is say “I know that you’re running this way, but here’s why you really want to run the other way instead.” A successful leader gets the crowd to go in a direction they weren’t already going. This seldom makes people happy in the short term, and in the long term they usually forget that someone else got them going in a better direction. So it has nothing to do with happiness.
What is good for us often doesn’t make us happy. Taking nasty-tasting medicine doesn’t make you happy – but it makes you healthy. Getting your vaccinations doesn’t make you happy. Eating healthy food doesn’t make you happy. You only get happier about these things if you can connect them with their long term benefits. In government, the long-term benefits ought to be the primary objective; make the investments that private industry won’t make because they won’t pay off fast enough. Maintain the military in peacetime so that it is ready to win when war comes. Build GOOD roads that will last and require less maintenance in the long run. People waiting months while a new 100-year bridge is built would be happier if a 10 year bridge was slapped together. At least for one election cycle.
I think this is part of the key problem with our country as it is today – we confuse the way things are with the way we want them and the way they should be.
While I can’t refute that many politicians and bureaucrats might claim this is the case, I don’t believe that is, nor should it be, the case.
The purpose of government is not to make us happy. The purpose is to make us safer, and to do things that we could not effectively accomplish as individuals. The actual reason governments do things is almost always to help a politician get re-elected or to allow them to do something that will make THEM happy. Of course, it probably helps your chances of re-election if your constituents are happy, but big campaign contributions help more.
There’s a quote attributed to Henry Ford, along the lines of “if I had asked people what they wanted, they would have told me they wanted faster horses.” The goal of government should not be to give people what they WANT. It should be to give them what they NEED – even if they don’t know they need it. What the people, inevitably, WANT is bread and circuses – for free, of course. What they need are standard weights and measures, clean water, armed forces, etc. Sure, if you ask people “do you want those” they’ll usually say yes – but if you just ask them what they want they’re unlikely to make the top ten.
Yet another quote is often told as a joke about a French political activist who sees a crowd running by. He runs after them, saying “I must see where My People are going so that I can lead them!” This is, in fact, a good way to get elected. It is NOT leadership. It is NOT what governments should do. What a leader would do is say “I know that you’re running this way, but here’s why you really want to run the other way instead.” A successful leader gets the crowd to go in a direction they weren’t already going. This seldom makes people happy in the short term, and in the long term they usually forget that someone else got them going in a better direction. So it has nothing to do with happiness.
What is good for us often doesn’t make us happy. Taking nasty-tasting medicine doesn’t make you happy – but it makes you healthy. Getting your vaccinations doesn’t make you happy. Eating healthy food doesn’t make you happy. You only get happier about these things if you can connect them with their long term benefits. In government, the long-term benefits ought to be the primary objective; make the investments that private industry won’t make because they won’t pay off fast enough. Maintain the military in peacetime so that it is ready to win when war comes. Build GOOD roads that will last and require less maintenance in the long run. People waiting months while a new 100-year bridge is built would be happier if a 10 year bridge was slapped together. At least for one election cycle.
I think this is part of the key problem with our country as it is today – we confuse the way things are with the way we want them and the way they should be.
Wednesday, February 6, 2013
From the Frontlines: Ursulmas
The feast of St. Ursul of Aquaterra was once again celebrated this year with violence and commerce.
Hmm. Maybe the mainstream holidays aren't the only ones becoming commercialized... :-)
This year for the first time in at least five years the main activities were split between two halls (so with the archery/thrown weapons we had three locations). It is my opinion that this didn't work very well; not enough cross-traffic between the fighting (heavy, rapier, and YAC were located in one hall) and the rest (merchants, A&S, etc.).
Lord Nigel, our current Baronial Bardic Champion, however, was an overachiever again and got a number of bards to perform in pre-scheduled slots on a 'stage' (backdrop, seating, and everything) throughout both days. Wanting to encourage this, I did a slot Saturday and one Sunday (30-40 minutes each) despite having the tail end of a cold. The audiences were small (about a dozen people at peak for either of my performances) but responsive. I hope we do it again next year.
House Weir (which has near-complete overlap with the Emerald Rain dance troupe) did a static display instead of dancing this year; everything a moorish woman would have had with her while traveling. Most of us pitched in something as well as the documentation for it (clothing, jewelry, food, furnishings, etc.). Somewhat to my surprise we won Best Static Display. I'd half forgotten there WAS a Best Static Display award!
I participated in the youth scavenger hunt, composed a story tying the various stages of it together (made a knife at A&S, went to Thrown Weapons to try it out, hurt myself and needed a Chiurgeon, etc.), and at the end each person got to make a quickie copy of a period board game (I chose an Roman form of tic-tac-toe). It was quite fun though I got a few odd looks.
All in all a good event, but I really hope we can re-unite the two major sections in one place next year.
Hmm. Maybe the mainstream holidays aren't the only ones becoming commercialized... :-)
This year for the first time in at least five years the main activities were split between two halls (so with the archery/thrown weapons we had three locations). It is my opinion that this didn't work very well; not enough cross-traffic between the fighting (heavy, rapier, and YAC were located in one hall) and the rest (merchants, A&S, etc.).
Lord Nigel, our current Baronial Bardic Champion, however, was an overachiever again and got a number of bards to perform in pre-scheduled slots on a 'stage' (backdrop, seating, and everything) throughout both days. Wanting to encourage this, I did a slot Saturday and one Sunday (30-40 minutes each) despite having the tail end of a cold. The audiences were small (about a dozen people at peak for either of my performances) but responsive. I hope we do it again next year.
House Weir (which has near-complete overlap with the Emerald Rain dance troupe) did a static display instead of dancing this year; everything a moorish woman would have had with her while traveling. Most of us pitched in something as well as the documentation for it (clothing, jewelry, food, furnishings, etc.). Somewhat to my surprise we won Best Static Display. I'd half forgotten there WAS a Best Static Display award!
I participated in the youth scavenger hunt, composed a story tying the various stages of it together (made a knife at A&S, went to Thrown Weapons to try it out, hurt myself and needed a Chiurgeon, etc.), and at the end each person got to make a quickie copy of a period board game (I chose an Roman form of tic-tac-toe). It was quite fun though I got a few odd looks.
All in all a good event, but I really hope we can re-unite the two major sections in one place next year.
Subscribe to:
Posts (Atom)
