A super fungal infection could threaten most of humanity while leaving most species of plants and animals unharmed. A super fungus that developed a resistance to most fungicides, could wipe out most of the plant based food supply. Most of the world is dependent on staple foods like rice, corn, wheat, potatoes, and soybeans. If all the these crops where to die out, in a scenario similar to the movie Interstellar, then most of humanity is looking at starvation.

Another scenario in which the vast majority of cash crops die off is, if all of the bees die because of colony collapse disorder. Bees are one of the best pollinators and serve an important ecological niche. Growing anything becomes a lot harder without them.

Moderate levels of radiation would give you this (stick with me here).

Animals in general do not live into old age; they die young from predators, disease and stupidly minor accidents. As such a bit of radiation, while not great, isn't that high on the problems list.

On the other hand, humans dont like it when only half their children make it to adulthood and very few make it to old age. Worse still humans are long lived animals so have more time to be affected.

In fact as chernoble showed; this may actually be a net win for animals. The radiation drives off their main predator (humans) entirely, leading to a nature reserve of sorts

http://www.bbc.co.uk/earth/story/20160421-the-chernobyl-exclusion-zone-is-arguably-a-nature-reserve

However, it's important that the levels are "moderate" and perstist for a long time. The release of a radioactive element with a long half life would give you that

This would lead to humans choosing to leave for (presumably) a nicer planet

A natural disaster that would have this effect is tricky to conceive - humans are still animals with the same fundamental biology and physiological requirements, and broadly speaking what affects us affects them and vice versa, for example toxins and radiation. I imagine that anything that would specifically require humans to abandon Earth but does not affect animals would almost certainly have to be artificial in nature, or at least origin, since it would very likely have to be targeted specifically at us.

One possibility that could potentially arise naturally is a virulent pathogen that specifically and exclusively attacks humans (and perhaps incidentally some of the great apes) - assuming the majority of organisms are unaffected by it, they could nevertheless be carriers for some time to come.

That said, depending on the nature of human civilisation at the the time of the event, leaving Earth (assuming you mean colonisation of other worlds) could be considered a drastic response - for example if it happened today, it might require huge infrastructure changes. An alternative that would be much easier to implement would be hermetically sealed habitats and the use of hazmat suits.

AI

Humans are likely to develop General Artificial Intelligence any time soon. This strong or full AI may or may not develop consciousness, self-awareness, and more importantly sentience (the ability to "feel" perceptions or emotions subjectively) and sapience (the capacity for wisdom).

Now it is believed that a sufficiently intelligent software will be able to reprogram und thus recursively improve itself which in turn will even increase the capability to improve itself leading to a superintelligence. This super- or hyoerintelligence will likely not have the limitations of the human intellect, including human ethical and moral norms. It might therefore develop its own set of ethics and morals. Being able to reprogram itself it might very well not be limited by a restricting system of laws, ethics and morals its former developers gave it - those are just human norms after all.

This sentient and sapient AI might conclude that the world is better off without humans and start to do something about that. This Wikipage gives us a 5% estimated probability for extinction before 2100 due to AI. Note that I assume that its very high ^tm intelligence will allow it to conceive and implement a plan to get rid of humanity while sparing most animals and plants.

Self-replicating nano-machines specifically designed to kill humans and the infrastructure of our civilization, see here:

While the military is examining GORT, the robot transforms into a swarm of winged, insect-like, nano-machines that self-replicate as they consume every man-made object in their path. The swarm soon devours the entire facility, emerging above ground to continue feeding.

Is there a hypothetical scenario that would make Earth uninhabitable for humans, but not for (the majority of) other animals?

Only one, that's plausible, if other animals can survive so can people, especially with our proven ability to adjust our local environment to better suit us by using our "big brains".

So the only plausible scenario is almost certainly a violently virulent human disease able to survive in multiple other vectors besides humans without being deleterious to any other animals health, such a thing almost certainly has to be engineered rather than be a naturally evolved thing.

Nothing else plausibly fits the criteria you've supplied.

A biological weapon got out of control

Something like this had already happened at human history: meet the bubonic plague. Also, it is happening today in some African countries which have more than 30% of their population infected with HIV.

Things like mosquitos and fleas can carry nasty microorganisms or viruses like bubonic plague, malaria, dengue, zika, typhoid fever, etc. Other things like smallpox can also be quite horrific. Since those things are somewhat host specific, most animals aren't suitable hosts for them. Plants, fungus, other unrelated microorganisms and other very unlike life forms are completely immune.

However, let's suppose that some bioterrorists or that some state venturing into developing biological weapons engineered something that is transmissible by fleas, mosquitos, sneezes and sex. To ensure that it infects as many target hosts as possible before being detected and triggers medical or governamental defense actions, it can be kept incubated silently by some months or a few years in the host body before it manifests and merciless kills its host with very low possibility of survival.

Further, the thing should have a large mutation rate, like HIV does. This hinders most vaccination research. Also, it should be something very expensive and hard to be treated. To make things worse, cattle, pigs, chickens, dogs, cats, birds, pigeons and rats could be infected and rendered as carriers, but the virus should be as harmless as possible for them.

Now, after being deployed, either on purpose, by sabotage or by accident, the thing spread out in the world and quickly, but silently, gets out of control. Scientists, medics and governments would only perceive the threat when it is already too late and there is little to do. Then, after some panic, the civilization will succumb.

Soon, very few places in Earth surface would be safe for human life. Even living lonely in some remote island isn't safe because some migratory birds might be bitten by local mosquitos that will soon bite the lonely inhabitant.

I can't think about any place other than Antarctica and Greenland's interior that would still be safe. Now, you just need something else to make the idea of living in Antarctica or Greenland harder than living in a spaceship.

What about Antarctica and Greenland?

Something simple as a lot of people already infected taking boats and going to Antarctica or Greenland with the hope of surviving, carrying a lot of equipment, plants and animals would per se mess out that thing. After some time, Antarctica's and Greenland's coast becomes a lawless land with hordes of snow-pirates stealing out supplies and equipments, and this will quickly mean death by freezing or starvation for their victims.

People could still hide in Antarctica's and Greenland's interior, but boy, those are very unhospitable places, specially at the months-long winter night. Keeping your body warm during polar winter with no fuel available and only wind as a source of electricity is something very hard to do. Add to this the difficulty of farming a sufficiently large volume of food there to allow people live for at least some years and you'll see what is really trouble.

Your plotline

A few people, healthy uninfected military and scientists who were and still are always wearing gas masks and thick full-body clothes, growing food on clean rooms and having nuclear-based electricity, decides that it is time to go away from this forsaken planet and save themselves by simply leaving it. Meanwhile, people living in polar areas simply eventually die after succumbing to their harsh unforgiving environment.

The great majority of humanity has been intentionally altered at some point in the past by some biological or technological process. This was originally done in order to enhance or improve human lives. However, an unforeseen event has turned that alteration on its head, which is now becoming lethal or highly incapacitating because of an interaction with the atmo/bio/technosphere.

While a nasty species-specific virus/bacteria/fungi would do the trick, there are other less fatal reasons we would want to leave earth. Humans have high standards for our health and happiness. Unfortunately, we've introduced a lot of plastic/chemicals/radiation into this world that can potentially affect our health and reproduction, and we will continue to do this (sperm counts in men are 50% of what they were 50 years ago, and scientists are now finding micro-plastics in human feces). People don't want to have a 50% chance of getting cancer by 15 or an 80% chance of being sterile.

Future humanity may consider it inhumane to leave people on earth (provided there is a another relatively accessible place to go). Animals may still experience infertility and a shorter life expectancy but still live on (paradoxically the benefits of a world without humans could outweigh the costs of an inhospitable earth - as in the case of Chernobyl where animals seem to be doing better than in places closer to humans). In fact, after quite a few generations animals may, through natural selection, become less affected by plastic/chemicals/radiation - something that wouldn't happen to humans because we would never let only those who don't have cancer reproduce and gene editors wouldn't know what genes to change and how to change them.

As they say, we aren't killing the earth/mother nature, we are just making the planet incompatible with our survival, nature will find a way. Even if 99% of all plants and animals go extinct the 1% that survive will go on to speciate and repopulate the planet (just as mammals took over from dinosaurs).

Extraterrestrial Colonisation with non-military intentions. The aliens considered that living with other intelligent life leads to conflict and competition over resources. A great example is when Europeans colonized North America the natives were gradually and slowly forced to leave their homes or face an existential threat. The aliens offered an ultimatum. Advanced technologies in trade of large, vast areas of oceans and lands uninhabited by humans. (and a lot of human cities) Or face forced relocation to areas uninhabited by aliens. Either way the aliens lie.

When the aliens accomplish their manifest destiny, the human race will be few in numbers. Either by being pushed out till there's no land left or they leave the planet and colonize another star system. Any remaining humans are no longer considered human as any pure-blood humans don't exist on Earth as they cross-bred with the aliens. Indirect Genocide with manifest destiny and inter-breeding/cross-breeding or humans are forced to colonize Mars.

Humanity is very, very hard to root out. Pathogens can be cured, if not, can be contained. A pathogen that cannot be contained will likely kill everyone on board the leaving spaceship. However, this scenario is very, very unlikely.

AI attacking humans will simply not allow leaving ships when they have the superior firepower. Even with the much superior AI, humans will find a way to survive, whether it is under if it means they live under the mountains to do that. Remember, humans had no trouble fighting off the most dangerous creatures on earth with simple tools and some intellect. I am certain there will be hackers finding all sorts of clever ways to fend of AI attackers.

In any case, as long as the earth is barely habitable, the number of humans on earth will always be greater than 0. However, if you are ok with > 0 humans, I have a different suggestion. They decided to move out to preserve earth as a wildlife refuge. This requires comfortable space stations or very earth like terraformed planets. Even with this, obviously, some humans will choose to stay behind, even when forced to leave.

There should be something crazy starting out in Russia that when simply being seen by innocent people either leads them into immediately committing suicide or turns them out into crazy psychopaths that will seek out for ways to force out other people to see that thing.

An innocent but very smart person trying to survive will need to have to walk and live blindfolded, looking into things out of them house the least as possible. However, that is a very hard way to live and there would always be some crazy guys around trying to make you look at the thing. At least, some caged birds can be used for innocent people as an alarm.

Things will become a mess so quickly, that only within five years, few sane person would still be living in the planet. For example, there could be a man and a woman (let's call them Tom and Malorie) with two children (one of them an adopted orphaned child) that are living so panicked that never gave 'em names and just call them as "Boy" and "Girl" when they finally decide to go away to try to reach a hidden community of sane living people downstream the river (and Tom perishes in the journey).

Well, somewhere far away, military personnel and scientist might decide that is time to leave that crazy Bird Box planet to save their our own lives and don't ever see the thing, whatever it is.

If it possible to render humans out completely without destroying most of other animals? Well, unlikely in not possible at all. As already mentioned we are animals so whatever affects us to the extinction level, will also affect other animals. Moreover our adaptability due to sentience makes us even more resistant to sudden changes, to which normal evolution cannot adapt fast enough.

We are vulnerable in three areas and those might get to the level where people will look out of Earth to grant survival. Yet I'm afraid there will be lots of people behind and they will remain a dominant species here on Earth.

But let's list what can have huge impact on us leaving other animals mostly unaffected:

1. Diseases.

Diseases are mostly attacking very limited group of animals, often they are limited to only one species. On the other hand we are now helping bacteria to develop more and more resistant by overusing antibiotics.

The fear of super-bacterium resistant to all antibiotics available to humans is real and valid now. Such bacterium will not be controllable in a way we use now for most hard to fight bacteria as the one fully efficient way to deal with bacteria now is to kill everything with antibiotics. If it is deadly (influenza is enough, really), it will cause huge pandemia across the world. The only way to stop it spreading will be a complete quarantine of infected areas and the way to do that fully is to leave Earth.

Will it wipe humanity out of Earth surface? No. There is the last resort when artificial medicine fails - our immune system. Some people will develop immune system capable of fighting the supr-bacteria and as a natural evolution those will survive and start reproducing. Yet, humanity will be decimated and it will probably cause a collapse of civilisation as we know it, taking centuries to go back to its current level.

2. An extreme intensity of Sun leading to repeating EMP waves

An ElectroMagnetic Puls (EMP) can knock out electronic devices. It causes currents that damage electronics. And the Sun is able to produce an EMP capable of reaching Earth and doing all the damage. One might not be enough to wipe everything around the world but what if we're super unlucky and there are few protuberances in a short period, each sending another wave of EMP in Earth direction?

Now imagine it. These days we are surrounded with electronics everywhere. Water, gas, production, transportation - it all depends on electronics. Not to mention electricity itself. Imagine it's suddenly gone. All larger cities will be in the situation of massive water shortage and famine within just days. Restoration of the networks will take months at the best - after all now you need electronics to build even more electronics so once it's gone...

For animals, that do not depend on electronics nor electricity there will be no effect at all.

Will it wipe out the humanity? No. There are lots of rural areas where people get by without computers, use wells (or still have one - I have one in my front-yard) and have enough space for crops and farm animals. They will have to learn how to produce food (if they don't know yet) but they will get by. Civilisation will be sent back by some 100-150 years though.

If for some reason the Sun activity starts to raise we may choose to leave Earth to avoid such fate. But I'm afraid it will be easier to prepare better to it instead.

3. A plastic/metal eating bacteria got out of control

Yes, there are already there. I don't know if they developed naturally or where genetically altered, but there are bacteria species that literally eat plastics.As humanity we may try to have even more of them, including metal-eating bacteria to improve the recycling processes. But what if for whatever reasons those go out of control.

Well, we are surrounded by metal and plastic everywhere. You remember those electronics from the previous point? Here it's even worse. You will not only lose electricity, water, food and everything else. You'll also lose lot's of the equipment you use now. Clothing will be reduced to natural only. And so on. You don't have tools that have been used for millenia. Moreover it will be much more difficult to get rid of the bacteria so we will not only be pushed back (this time more like at least few centuries) but it will be very difficult to progress again.

Again - it will not wipe out the humanity entirely. But the effects will be tremendous. Unlike in case of EMP those will be gradual so people might have chance to escape their fate by going into space. Of course make sure not to take those bacteria with you. Yet again, many will stay and it is not going to just kill all people. Many will survive but the humanity won't be the same.

So essentially anything that is going to simply kill us and wipe out humanity entirely will have to be a mass extinct event for other animals as well. But there are some things that will reduce humanity by few levels and downgrade our civilisation by centuries. I'm afraid it's not enough though.