When photo analysis software misleads

I long ago learned the hard way that we can become victims of our technology. I found that facial overlays are useful when I am careful about the eyes, but then found that there is technology available that can do overlays, as above, and get very bad results. I was doing overlays for a while, and then reverted to my old technique, the face chop. It is surely primitive compared to what iPhones and Androids are doing with facial recognition technology. I will never forget standing in line waiting to get on a plane in Atlanta as we were quickly ushered aboard, a camera instantly recognizing our faces and correctly identifying us. Gadzooks! What I do is primitive.

Nonetheless, my method, the face chop, has some rigor to it. It is based on the simple premise that our skulls are formed by our late teens and do not change in shape. The bone structure remains unaltered over time even as our skin ages, sags and wrinkles. So I use a common measurement, the distance between eye pupils as a standard when doing comparisons. Where this technique fails is in undervaluing the distortion caused by head angles. So I learned to be very careful and not to over-analyze. Head angles got me into a heap of trouble, seeing twins where there were none. I found that the face chop is indeed useful, but it does not take the place of basic intelligence.

The facial overlay above between Gates and Zuckerberg is done by forcing the head sizes to be the same. It is therefore not objective. The face chop below sets the eye pupils at a common distance, and these two public geniuses are easily seen to be two separate private morons.

Of course, I cannot be in the same room with them, and to be truly objective here, I would need to find a way to judge skull size by objective means, such as a photograph of the two together. There are a few photos like that, but they would then have to be of equal distance from the camera so that I could run some calculations using Microsoft Paint and measuring pixels.

I did so with the photo above, and here is what it told me: Gates is a tad taller than Zuckerberg, and Zuckerberg’s head is bigger. To me this photo indicates that the two men differ quite a bit in head size. That is also what the face chop told me, but the face chop is not an accurate indicator of head size.  This is just luck.

Here is an example of where my technique is useful. Janis Joplin was a set of twins. I know this because I spent hours on end staring at her photos, confused that I could not get early and later photos of her to line up. Once I discovered twins, everything fell into place.

That is how I learned that misdirection plays such a large role in media. Joplin would appear in public with big hair and eyeglasses, which served to distract us from looking closely at her facial features. They did not kill Janis, simply reassigned her to a new public role, and sure enough, we found the twins still pretending to be one person.

Here we have both of the Joplin twins pretending to be “Amy Goodman,” Democracy Now!’s lead anchor. I think the reader can easily see two people, but if not, there is an easy way to tell. When the twin on the right smiles, she bares her teeth. The one on the left is demure, never showing her teeth. The one on the left posed nude – I won’t show the photo here but you can easily find it. My god that had to have been difficult for her! The one on the left was the stage performer. Personality-wise, they are night and day.

I ramble here, did not sit down to revisit the Joplin twins. The point is to be wary of software that does facial overlays. I found, for instance, that I could do a near perfect match between singer John Denver and actor John Ritter (both of whom likely faked their deaths). I could have easily made that happen by merely changing the head sizes to force a fit, as was done with Gates and Zuckerberg. But I did not do that – I set the pupil distance at the standard one inch apart, and let the chips fall. Here is what I got.

Denver Ritter

What does that tell us? There is something going on in the world of entertainment – some process by which near-clones are birthed by different parents. Speaking of parents, here is Denver with Stephen Parent:

Denver Parent

Parent, supposedly murdered by Charles Manson, went on to become Richard Branson, the billionaire. Tyrone spotted that one. The technique is useful.

These days, when computers can generate faces out of nothing and make them look so real, perhaps the art of facial comparisons has seen its heyday. I don’t do much of it anymore, and find it most useful, as with Gates and Zuckerberg, in deciding that two people are not at all alike.

25 thoughts on “When photo analysis software misleads

  1. Perhaps it means that the cloning process has been going on for quite some time – and there are only so many different clone bodies to choose from so there are bound to be similarities.
    ?????????????

    LoLz
    .
    .
    .
    ***hmmmm…

    Like

    1. Hmmm … after I wrote this I wondered why I assume Parent became Branson – I guess because Parent’s death was faked and he needed to be somewhere. But he and Branson could easily be two of the same mold.

      Like

  2. My take on the problem with the Zuck-Gates 50/50 meme is much simpler. It would be like someone saying they blended 50% vodka and 50% orange juice, then they blended 50% orange juice and 50% vodka. Whoa…same result! Color me shocked! Inherent flaw in the logic of the meme there, and it is trying to take advantage of our pre-established skepticism of Zuck and Gates.

    One thing that merits mention on the whole face-chop concept is that it is absolutely critical the face be at the exact same angle and relative distance. Even very minor rotations of a skull in 3 dimensions would have huge ramifications on the face-chops, which as you say are primitive in nature and should only be used for guidance.

    There is a reason that we are all forced to stand at a set distance marked off on the floor at the DMV and maintain a forward-facing posture. This process results in a totally standardized photo of all citizens, which I am sure is feeding tons of facial recognition software. Such standardization benchmarks are inherently critical in any kind of facial analysis. We ought to be careful with the face-chops that we are never comparing a close-up to a distance shot or using photos tilted in slightly different ways.

    When I have previously joined in facial analysis here, the most time-consuming aspect for me was finding two photos of the targets at the exact same angle and distance. The cell app I had used, a free app called Mixo, would automatically standardize the head sizes for comparison after I found the perfect two photos. That way we eliminate the possibility of a non-match caused by large skull versus small skull (but actually containing identical facial ratios/symmetry). This kind of standardization is what would be needed to really go far with the concept.

    Some of these people may indeed have had their deaths faked, but that does not inherently mean that they are now replaced by this-or-that celebrity…simply because we can find a very good facial ratio match.

    I know you have said your basis is setting the pupil distance, but I am seeing a photo of Janis Joplin here in this post that does not seem based on pupil distance. Just a word of caution…it is ground that we need to tread carefully on to not run wild with our basic assumptions. One example: I would be interested for a deeper dive on Amy Goodman, as I only see one woman there.

    As you know, my favored explanation is that there is an already active Gattaca-style genetic engineering program among elites. If they want their child to be attractive, such things can be made to happen through genetic modification of facial ratios and symmetry (among other things).

    My favorite new tool for celebrities is to use Getty photo service. You can access free ultra-high definition photos of celebrities (by simple search in a search box) which you can then sort chronologically. If you are looking for doubles or replacements, that is an ideal place to go. You should be able to find photographs at such a high resolution that you can compare moles, freckles, scars, etc. and attempt to pinpoint an exact time-frame for replacement. I studied the rapper Eminem in that way with some really baffling results.

    Like

  3. The problem of distance is solved by standardizing the pupil distance. That part works quite well. I’ve done it thousands of times and have come through choppy periods where I relied too heavily on it and ignored angles and all of that. I’ve developed an eye over the years, and most importantly, never fudge results. It either gives up information or it does not. Without the pupil distance standard, as seen with Gates Zuck, you’re compromised at the outset, as you can manipulate head size and force results. I’ve actually done that, increased John Denver’s head size to force him to align with someone, but I was transparent about it, not fudging.

    Goodman is just skipping over the work that we did years ago, hundreds of images. I could have run them all again, but I assure you the two Amy Goodman’s are the two twins. The one on the right only rarely appears, but she was the key to it all. Once we found her photo, all of the others fell in place. If you see two different women of younger age above, then I have images of them with older women that match up very well. I suppose I have to go back and do that again, before Janis Joplin disappears from memories.

    Gattica aside, I’ve found not just good looking people, who come in all forms, but people whose facial features are amazingly alike. Some of the resemblances, as in Judy Woodruff / Delila, or Jennifer Lawrence / Helen Murren, are stunning.

    Anyway, it’s late and I am tired, but we should discuss this further. As I say, I don’t do much of it anymore. but I’ve done enough of it to know that it is not random. People do not line up facially by accident. The odds heavily say not. And it is coincidental logic, not just incidental. What are the odds of the same placement of nose on two people? 1 in 1, it can easily happen. What about nose and lips? No longer 1, but some other number, perhaps 1 in 5, and then the eyes, the shape, of the skull, with each attribute the odds against perfect alignment grow larger. With my face in Atlanta, they are using an algorithm that must incorporate some of this. Warts and moles are useful, and I found with Andy Warhol Stu Sutcliffe some precise placement, but there you have to rely on photo honesty, as stars are usually heavily made up and photos doctored to remove blemishes. But basic features are less susceptible to manipulation.

    Like

    1. I really don’t understand at all what you mean by a standardization of pupil distance. That is not what I see at all in the photo of Joplin above, and how would that solve the problem of a close up versus photo at a distance? One would be huge and the other tiny, regardless of how you set the pupils in relation to one another. I must be missing something there. There has to be SOME kind of standard. Please help me understand what you mean about pupil distance, especially like in the middle Joplin photo. I don’t follow.

      Like

    2. Certainly you must be doing something to choose two photographs where the skulls appear to be the same relative size? This is what I’m asking. Help me understand the resizing. You talk about setting the pupils at a certain distance (again, I’m confused about the Joplin photo where her eyes are totally misaligned), so help me understand why that works to somehow make this a fair comparison. I must be missing something because I can’t see what you mean.

      Then what you call “forcing it” is actually what I think should be seen as the most precise way to do this. I think the most advanced facial recognition software in the world forces it in this way. That is, viewing this from the perspective of the facial ratios. Skulls will not align if they are different sizes, but the facial features might actually display completely identical symmetry to one another. So I suppose you are seeking skull alignment here rather than facial alignment? From what I’m gathering, if the skulls are different sizes you would consider that a non-match almost by default.

      I’m lost on what you mean by pupil distance with the Joplin photo above and without a framework for what guides you to choose each photo or why the pupil distance makes it all equal.

      Like

      1. The Joplin photo is not aligned in that manner, and was merely put ip to show that the two women are obviously not the same person. The pupil distance work was done in the original post years back. I guess I have to bring it forward now.

        I wonder sometimes … I can easily see two McCartney’s at a glance now because of my extensive work on the subject, and the same with the Joplin’s. Do I need to continually reinvent the work? Anyway, I’ll revisit the subject.

        Like

    3. For example here, why do you leave Zuck larger than Gates? It seems that making Zuck’s head smaller would cause the nose and the mouth to come closer into place. Why did you accept the photos of these two heads at these particular sizes? Without an understanding of the distances involved in each of the photos and/or a knowledge of how big their’s skulls actually are, I don’t understand why you could feel confident about the relative sizes to use for your comparison…pupil distance set to a certain point or not.

      Like

      1. Do you resize until their eyes are similarly sized, and then consider it about equal for comparison? That would make some sense to me. It’s a tricky thing and I realize you have lots of experience with it. I think there’s something lost in translation where you say pupil distance that perhaps if you described it differently it would make more sense to people who don’t do these comparisons for themselves. You could be talking about tiny pupils from a person at a distance. You set the pupil a specific distance, then what? You size the photo up until they seem the same relative size?

        Like

        1. I set the pupils one inch apart, and do nothing else. If you do that and it is the same person, the other features magically align. You mentioned distance from camera, but aligning pupil distance cures that problem. You mentioned head angle – that affects ears heavily, but not central features.

          I never adjust the size of the skull other than to set equal pupil distance in two photos.

          Like

        2. So here are the Joplin “twins,” that is, the first will show two photos of them each claimed by other sources, not me, to be “Janis Joplin. If they are the same person, they should align precisely. They do not.

          The chop below is a photo claimed by “Amy Goodman” to be her as a younger woman, and one of “Janis Joplin.” I got perfect alignment.

          What does this mean? The implications were large – it started with the discovery that dead rock star Bobby Fuller was still alive, and going by the name Bill O’Reilly. I did not instantly understand all of this, and still do not, but came to think that rock stars of that era, 60 or 70 said to have died, were merely reassigned. I found a lot of them still at work, and stand by the work.

          Like

      2. Zuck’s head does not align with Gates when pupil distance is set at a common distance, and that is the point. If these two were the same men in different roles, then their faces would align based on pupil distance because that measurement does not change as we age. The comparison above shows this. The technique not only gave me evidence that Janis Joplin and Amy Goodman were identical facial plates, but also that there were two Joplins. Again, I’ll have to revisit.

        Like

      3. I was attacked on another blog by a man named “JC” who ran an image of me with John Candy noted how much alike we looked. All he had done was to put two photos side by side, and adjust the size of the skull to make us similar. That’s bad technique, as there is no gold standard. Pupil distance is a gold standard and can be used for comparison purposes. No, Zuck’s head is not that much bigger than Gates’s, but what the technique showed was that the two man were not the same person, and that Tumblr’s work was defective. That’s what a face chop does. It can show you that two people might indeed be the same person (Joplin/Goodman) and that they are two different people (Joplin twins). But if you arbitrarily adjust skull size, you not nuthin’.

        Here is me in two photos thirty years apart using the technique. I do not fudge. All I did was set the pupils at common distance and let the chips fall. Note how my smile is alike, as is my head angle. This is not uncommon. I noted with young Paul McCartney and old one (Halliday) that to do the comparison I had to tilt each head 4 degrees.

        I have repea3tedly said over the years that anyone can do this, but none do. It’s not complicated, and can be used dishonestly, as by Dallas Gold something or other. But I use it in all honesty for objective results.

        Like

  4. I have seen several things (here and here and here and here and here) about what a difference focal length makes in the shape of faces.

    Mark, does your technique control for focal length? How could you know what focal length a photographer used for a given photograph? And if you don’t know, aren’t you mixing apples and oranges for your face-chops?

    Focal length changes pupil distance. I bet if you did face chops on some of the faces in the links, you would conclude that the same person is two or three people or more …

    In all of your posts about twins and zombies, I have never seen you address the issue of focal length. A few commenters have, but you have not responded to that specific point.

    Could you here give us your thoughts about the way focal length affects your match-ups of split faces?

    Like

    1. Focal length can cause distortion. I repeat, I never fudge anything, and if I do not get alignment, I conclude that the work is pointless and I move on. I have done this stuff hundreds upon hundreds of times, and for the most part, I got nothing concrete. You only see those results where I am giving evidence that two people are not the same (Zuck/Gates) or are the same (Goodman/Joplin).

      Much of what those links above show is deliberate distortion, but honestly, you act as if I do not have eyes and just blindly compare faces that are noticeably different in shape. I took your first link and did the usual, putting both photos in Photoshop to level the eyes, and then over to MS Paint to adjust for pupil distance. For the photo on the left, the adjustment was 114%, for the one on the right, 110%, that is, distortion was 1/32 of an inch. The result …

      The features align. Yes, I note the longer beard on the right, and if this were a real deal I would have to take that as evidence against these being alike, because you can be sure commenters would take note. I cannot just overlook this stuff. But here they told me outright these are the same man. The only thing proven is that focal length causes distortion.

      I was quite surprised that there is a lack of understanding about the importance of pupil distance, that once our skull is done growing, that that distance does not change for the rest of our lives. That makes this tool useful, but at no point does it override the need to use one’s brain in the analysis. Even where there are only tiny differences, it is still enough to discard the project, as it would not be convincing anyway. People who critique my work are very keen to tiny differences and oblivious to the bigger picture.

      The only time I have seen the shape of a skull change over time was Stephen Hawking, not the imposter they inserted in the 1980s, but the original guy. His head seemed to swell and get heavily elongated over time. That made it impossible to offer any evidence that he was the same man over time, or that they switched him out when he died in 1985.

      PS: I had to delete and re-insert this comment because I edited it, and when I do that, any image contained in it vanishes.

      Like

        1. Longer answer, Yes. It is automatic with pupil distance. If things don’t line up and there is distortion due to focal length, things don’t line up and I do not use it. I mean, photos on the Internet are not labeled as to type of lens used.

          I take as a general inference that I should not be doing this kind of work, that all of my discoveries over the years are suspect. I disagree emphatically.

          Like

      1. I really wish you would do an explainer on what you mean by pupil distance, because without controlling for other factors the pupil distance doesn’t really mean anything to me. I’m guessing you do have methods of controlling other factors that are just getting lost in translation.

        What do you mean by pupil distance? Explain the process there, because I imagine it must also include some kind of method to identify two photos of a person at a similar apparent angle, distance, focal length, etc. Without that, what good is setting the pupils at a certain distance? Angle, distance, would totally throw that all out of whack. The size of the skull could be made to seem identical in two photos, but if you had them in the same room it would be clear that the skulls are totally different in size.

        All I’m saying is that I have always held that standards need to be upheld and I’ve never understood what you imagined made pupil distance such a universal controlling factor. Help me out there.

        Like

        1. This actually helps because it’s not what I had pictured! This IS the sort of resizing and standardization that I had envisioned. It should make sure that the heads are the same relative size.

          I still think the angle is huge but this helps me understand. I thought you meant an inch between the eye of photo 1 and photo 2, not to resize the heads of each so that the eyes are a standard distance (an inch) apart before aligning them. Thank you!

          Like

        2. Years ago I read that our skulls are fully formed by our late teens, and we have two eye sockets that will not change in distance from one another for the balance of our lives. I’ve shown this to be true countless times, including with myself, as shown above. The pupil is the black dot in the center of the eye. Assuming a person is not cross-eyed, the center of each pupil will be the same distance apart throughout their adult lives.

          Other aspects of ourselves change, but our basic skull structure does not. We therefore have a standard by which we can compare photos and judge whether we are looking at the same or a different person. MM prides himself on being an artist and doing facial comparisons without any mechanical aid. I like to have some assistance, and found that the face chop is the best tool of all.

          The photo above is taken from Maarten’s post, and I worked with it earlier. I grabbed it as a PNG and took it to PS to straighten the eyes horizontally. I then took it to MS Paint, which has a feature called “resize” wherein I can adjust the entire photo vertically and horizontally at once to set the pupil distance at once inch. In the phooto I have a transparent ruler. Say that tech photo us smaller than needed, usually the case, say that the eyes are 19/32nds of an inch apart. I increase the size of the photo by 168%, and then fine tune it a bit to get exactly one inch.

          I ten do the same procedure with any photos I am comparing. I then grab half the face (another MS Paint feature) and paste it over the other, and align the eyes for the two photos being compared, and that’s it. If all of the other features do not line up, then it is a no-go. You say head angle matters, and it does, but not as much as you would think – that affects the ears mostly.

          If all the features do line up, there is a strong likelihood we are looking at the same person.

          This is the best way to eliminate bias. There are other means, but I ve found none so reliable. And as I said, I do not fudge or attempt to do anything other than comparisons by this method. I’ve tried the ear, without much success, and things like distance between eye and chin, none were reliable. I will say, however, that last week I looked at four photos of Brandon Teena with others in that group, and noticed that his/her head sized varied greatly from one to another, leading me to conclude that his/her image had been Photoshopped in.

          I feel like I am defending my life today.

          Like

        3. PS: This technique says nothing about relative head size. I have said repeatedly that I cannot know that without being in the same room. When someone has narrow-set eyes and I blow up the photo to get the one-inch pupil distance, the head blows up too. That’s why Zuckerberg above appears to have such a big head – his eyes are narrower than Gates. That’s all that means. That’s why I ran the photo of the two of them together, showing that Zuck does indeed have a larger head. But that was just coincidence. I did not prove that.

          Back when I was working with John Lennon, I had a picture of two boys who looked much alike, with one having narrower-set eyes than the other. Thinking twins, I messed with various photos of adult Lennon, and sure enough, some of those photos when blown up to one inch were much larger heads than others. I gave up, however. I could not make it stick to my own satisfaction. I am still somewhat intrigued that there might have been two of him, just as with the McC’s. Intel loves twins.

          Like

    2. And, by the way, take a look at the face chops above of Joplin/Goodman and of me in photos 30 years apart. I do not fudge anything ever, it either works or not. In those two instances it gave very clear evidence of the possibility (Goodman) and certainty (me) that we are dealing with the same person.

      Like

  5. Since we’re talking photo analysis, what do you all make of the photos in this story (scroll way down)? They show minor celebrity John K with teenage girls, now middle aged women who accuse him of being a sexual predator.

    I found them very plausible at a glance, but when I zoomed in and studied them I began to see signs of cut and paste. I find it hard to be absolutely certain though.

    There are a couple more that come up in a google image search, one of which I do strongly feel is fabricated.

    https://www.buzzfeednews.com/amphtml/arianelange/john-kricfalusi-ren-stimpy-underage-sexual-abuse

    Like

      1. I actually started to write one but realized it would take several hours, including marking up the photos. But maybe I’ll write up a short comment with just a few stronger points highlighted.

        Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s