In June, the US government published a long-awaited report into UFOs. Although the report did not, as many had hoped, admit to the existence of little green men, it did reveal that not only were objects appearing in our skies that the Pentagon – which controls the US military – could not explain, but some clearly pose “a safety of flight issue and may pose a challenge to US national security”.

The Pentagon also revealed that it has been taking UFOs so seriously that in 2007 it discreetly set up the Advanced Aerospace Threat Identification Program (AATIP), which has been gathering data on Unexplained Aerial Phenomena (UAPs) ever since.

The unclassified version of the report (there was also a classified version seen only by US lawmakers) found “no clear indications that there is any non-terrestrial explanation” for the sightings. But neither did it rule it out. The report offered five typically mundane possible explanations for the UFOs and, crucially, one catch-all “other” bin.

It’s that “other” bin that has arrested the attention of stargazers and conspiracy theorists. If the US military has been quietly and seriously investigating UFOs (or, as the Pentagon would have it, UAPs) since 2007, and if the Pentagon’s official report cannot rule out the existence of extraterrestrials, is it time we looked again at claims of close encounters and the people who have made them?

Enthusiasm for UFOs and ETs has permeated popular culture ever since a US air force balloon crashed near Roswell in 1947. Conspiracy theorists confused the balloon for a UFO; the US government did a lousy job debunking those claims, and they quickly captured the public’s imagination. Fast forward to 1961, when Barney and Betty Hill told the world’s first alien abduction story.

Andrew Abeyta, professor of psychology at Rutgers University, co-authored We Are Not Alone, a study into why some of us want to believe in aliens. Abeyta explains that belief in aliens is akin to religiosity: unfounded beliefs in unfalsifiable ideas, which require a leap of faith. “People have a need to feel like their lives are meaningful, and these beliefs might suggest that there’s something bigger out there; there’s something more important going on,” Abeyta says

I tell Abeyta about an interview I carried out with a young man in Florida. The man, who did not want to be named, described an ambiguous close encounter that took place during his sleep. When I asked him what he preferred the truth to be – a real encounter or merely a vivid dream – the young man said he would prefer it to be true because that would mean he was “special”.

“I can imagine being a protagonist in an alien-abduction story seems pretty meaningful, like a meaningful achievement, an accomplishment,” Abeyta says. That feeling of specialness plays an important role in these stories. “Feeling like your unexplained experience is a result of an alien abduction just seems more exciting and more important than a natural explanation.”

Still, the topic of alien encounters remains sensitive. I discovered just how sensitive when author Whitley Strieber, who some claim was abducted by non-humans in 1985, terminated our call after learning that I had not read his books. In a subsequent email, he wrote: “I don’t know if I was abducted by aliens or not. The whole point of my work is to describe what happened to me and attempt to understand what it was. I was turned into ‘alien abductee Whitley Strieber’ by the media. That is not my position.” He added: “You are lost in space when it comes to this subject, my friend – all of you.”

After I got off on the wrong foot with Strieber, though, he did come back and introduce me to highly decorated former US navy cryptologist Matthew Roberts. He was stationed on the aircraft carrier USS Theodore Roosevelt when fighter jets recorded the infamous “Gimbal” and “Go Fast” videos of unexplained objects off the Florida coast during 2015, which went a long way to prompting the Pentagon’s UFO report.

Now retired from the military, Roberts is unmoved by the debunkers. “These things are picked up by multiple sensors that are sometimes from different manufacturers, so to think that they would all be glitching in the same way at the same time would just be impossible – it just doesn’t happen that way.”

Mick West, a science writer and video game programmer turned conspiracy-theory debunker, offers his own, more down-to-earth explanations for the objects: arguing that mundane things – tech glitches, camera glare, balloons and birds – are more likely than aliens.

However, now even the Pentagon has conceded there’s more to UFOs than that. In its nine-page report it states: “Most of the UAP reported probably do represent physical objects given that a majority of UAP were registered across multiple sensors, to include radar, infrared, electro-optical, weapon seekers and visual observation.” In other words, there was something out there and the images were not technical glitches. I ask Roberts about a theory put forward by West that the Gimbal object was glare caused by a nearby aircraft. “All aircraft – nationally, internationally – have to broadcast who they are. If they’re not broadcasting that, that’s very unusual. Mick West, bless his soul, he has never been in the military,” he says.

Roberts explains that, after the 9/11 terrorist attacks, unidentified air tracks escalate very quickly. “It will go to the captain, it will go to the admiral, and they’ll want to know what that is because the thought would immediately be: ‘Is this a commercial airliner? Has it been hijacked?’ We’re not as incompetent as Mick West would have you believe. If something is unidentified, it absolutely has to be identified immediately.”

Despite the debunkers and proliferation of more mundane explanations for UFOs, reports of close encounters have persisted for decades. Terry Lovelace, a retired assistant attorney general in Vermont, USA, and author of Incident at Devil’s Den, kept his abduction to himself for 40 years due to fear of losing his job. He had a close encounter in 1977 while serving in the US air force.

Lovelace, now 67, was on a camping trip in Devil’s Den state park in northern Arkansas with a friend and colleague named Toby when things got strange. They were sitting around a fire, struggling to chat over the din of buzzing crickets and croaking tree frogs before everything went quiet. “That sounds kind of clichéd – out of a movie – but that is exactly what happened to us,” he says.

Three bright lights appeared on the horizon and moved in their direction. When the lights were overhead, they could see that they were emanating from a black triangular prism as wide as two city blocks.

A blue laser beam darted over them, which Lovelace thought was scanning them. When it shut off, they became sleepy. Next thing, he woke and saw Toby peering out of the tent. The triangle was hovering above what appeared to be a dozen children standing in a meadow below them. “What are these kids doing out here in the middle of the night?” said Lovelace.

“They aren’t little kids. Don’t you remember they took us and they hurt us?” Toby answered.

Lovelace says the moment Toby said that, fragmented memories of being inside the UFO flashed in his mind. Years later, hypnosis helped him fill in more blanks and he recalled actually encountering creatures while inside the UFO.

For some, the fact that the Pentagon has finally admitted it cannot explain the behaviour of the objects may have been a surprise but, for PC Alan Godfrey, 73, it merely proves what he already knows.

On a windswept and wet West Yorkshire evening in November 1980, Godfrey was in hot pursuit of a herd of escaped cows in Todmorden’s housing estate. Instead of cows, he stumbled across a giant levitating diamond that would change the course of his life. Godfrey’s close encounter with this UFO went viral worldwide and transformed Todmorden into Britain’s Roswell.

Godfrey, a no-nonsense Yorkshireman born and raised in Oldham, is long retired from the force but still recalls the events of that night when he came face to face with the peculiar object – a diamond-shaped aircraft hovering 5ft off the ground while spinning on its axis.

He just had time to sketch the UFO on his notepad before he was blinded. In his next moment of conscious awareness, he was sitting in his patrol car. The UFO was gone. “I got out of the car, looked at the road surface, and it was like a whirlpool,” he says. The UFO’s rapid revolutions had arranged the dead leaves, twigs and other debris into an autumn-themed spiral.

In the aftermath of his encounter, he had visits from the Ministry of Defence, correspondence from a Russian scientist and interest from the world’s press. He even underwent hypnosis to uncover memories of his abduction.

Godfrey was ridiculed for years – many who claim to have had encounters with UFOs are reluctant to go on the record for fear of the same treatment – but things are changing. High-ranking government officials such as Christopher Mellon, a former US secretary for defence in intelligence, and Luis Elizondo, former director of AATIP, insist that there are aircraft in our skies that don’t obey the known laws of physics. Even Barack Obama has gone on record on the subject, talking to CBS this year: “There’s footage and records of objects in the skies, that we don’t know exactly what they are, we can’t explain how they moved, their trajectory. They did not have an easily explainable pattern. And so, you know I think that people still take seriously trying to investigate and figure out what that is. But I have nothing to report to you today.”

When it comes to abduction stories, sceptics will say these encounters are either hoaxes or accounts of vivid dreams or hallucinations. Christopher French, emeritus professor of psychology at Goldsmiths, University of London, has spent years studying the paranormal and argues that sleep paralysis is a better explanation for many of these stories. “In some cases, you get associated symptoms, and they include a sense of presence; a very strong feeling that there’s something in the room with you,” French says. He adds that sufferers might hallucinate and “see strange lights moving around the room or strange figures or shadow people”.

That doesn’t fit for Godfrey’s story – he was driving and on duty at the time. “I think in Alan Godfrey’s case, he was sleep-deprived; he had been on duty for a long time. The most likely explanation is some kind of hallucinatory experience due to tiredness,” says French. What about the story he told under hypnosis? “The thing with hypnotic regression is that it is one of the best ways known of generating false memories. If you go for hypnotic regression expecting to recover memories of alien abduction, there’s a very good chance that’s what you’ll get.”

But Nick Pope, a former UFO investigator for the Ministry of Defence, is not convinced and thinks that Godfrey is genuine. “He had a lot to potentially lose by coming out with this and yet stuck to his guns.”

Doesn’t a hallucination explain what he saw? “I get that people do have hallucinations, but they tend to be the result of either mental illness or some sort of hallucinogenic substance, and this guy was on duty and was, by all accounts, rational. And so those explanations don’t seem to apply – I’m stumped when it comes to that particular case. Ask yourself: how many times have you been tired and come to the end of a long day? We’ve all been in that situation, and we don’t suddenly construct bizarre narratives about spacecraft and aliens.”

Is it time to start taking these stories more seriously? “I’m not saying that I believe it’s literally true that these are alien spaceships,” says Pope. “But at the very least, these people who were previously disbelieved and ridiculed should be listened to and given a hearing.

“For everyone who tells you these people are attention seekers after fame and fortune, I would say, ‘What fame? What fortune?’ Who outside the UFO community has heard of Alan Godfrey or Terry Lovelace?”

Does Pope think ETs are among us? “I don’t know. I am certain that they are out there, but whether they’re down here or not? I don’t know. I think it’s much more likely that we’re dealing with unmanned probes.”

If not hallucinations, equipment glitches or mistakes, many will say black ops, conducted by the US, China, Russia, or other militaries, are a more plausible explanation than aliens. “I accept that most military personnel won’t have sight of every single black project and, therefore, won’t necessarily know about every secret prototype, aircraft or drone that’s flying,” says Pope. “But the military and government, and the intelligence community have a pretty good idea of roughly where the ceiling is in terms of technology. So, when these expert military witnesses describe the sorts of speeds, accelerations, manoeuvres that are reported with these sorts of incidents, I sit up and take note.”

Whatever one thinks about the veracity of these stories, many of the people who tell them believe they are real, and some suffer from severe mental illness in the aftermath. Chris French says the levels of psychological arousal in people living with PTSD go “through the roof” when they’re asked to retell their stories. “If you do the same thing with the alien abductees, you get the same thing.”

Lovelace’s night in Devil’s Den changed his life and the life of his friend Toby. The US air force got wind of their ordeal and, per military protocol, separated and reassigned them. Lovelace ignored his orders and visited Toby to say goodbye. “Toby was falling apart,” Lovelace says. The two embraced. Toby said: “It happened, didn’t it?” “Yes, my brother, it really happened. You’re not losing your mind,” Lovelace replied.

Lovelace has suffered enormously since that night. “I’ve had 40 years’ of nightmares. I still have a phobia of crossing open ground. I still sleep with a light on and a gun beside my bed.” But he feels vindicated by acknowledgments made by the US government, military personnel and Obama. “I’ve got a long list of people that I’m going to email and say, ‘I told you so.’”

For Godfrey, it’s 40 years too late. He is adamant about what he saw that morning in Todmorden. “I’ve had all sorts: you fell into some sort of trance when you were driving – all that shit. No, it was real. It left debris on the road – my headlights were reflecting off it, as were the blue lights. This was a real incident. I didn’t need the Pentagon to tell me there are things out there. I know what I saw that night was real, nuts and bolts. If I’d got out and thrown a brick at it, it would have gone, ‘Clang!’ It doesn’t change what happened to me and how I was treated back then.”