I have a friend trying to make a hybrid between a celestial and a demon in 5e. I know the Aasimar exists (human/celestial) and I know the Tiefling exists (Human/demon) but what if you cut out the human middle-man. What would this creation be called? Is it RAW?

It seems like an incredible being, but it seems as though it might be really overpowered...

This type of creature is not described in the rules

After sifting through all of the celestials and all of the fiends, it appears there is no mention of a pair that could procreate in such a way to create a part-celestial/part-fiend individual. You could, however, create such a creature using the Dungeon Master's Toolbox in the Dungeon Master's Guide where it details how to create custom monsters and custom races (both of which could be useful in your friend's endeavor).

What we can find for celestials

In 5e so far, there are very few celestials mentioned in the rules. Even still, there are some creatures that seem promising. The only celestials that mention a mechanism for reproduction are the Couatl and the Pegasus (both found in the Monster Manual):

Couatl procreation is described as

...a beautiful and elaborate dance of magic and light, which results in a gem-like egg from which a new couatl hatches. The parent that sought out the mate raises the newborn couatl and instructs it as to its duties, so that it can complete whatever task the parent leaves unfinished.

I could easily see a fiend that reproduces through infecting the eggs of non-fiends. Such a creature might infect a couatl egg and the resulting creature may share some of the features from the original parents and the infecting fiend.

And the Pegasus entry simply states

Pegasi mate for life, build their nests in hard-to-reach locations, and give birth to live young.

This is pertinent as a nightmare (a fiendish mount) is created by

the torturous removal of a pegasus’s wings, driving that noble creature to evil as it is transformed by dark magic.

It is unclear if this nightmare would still be able to reproduce with their former mate, and what would result in such a union (if it were somehow facilitated).

What we can learn from fiends

There are a number of fiends that are born from the union of fiends and humanoids. Any of these creatures may resulting in semi-celestial origins if the humanoid shared celestial blood. The features of this type of creature (such as whether they remain celestial in any way) are also unclear. You can find a list of humanoid-spawned fiends below:

MM = Monster Manual ; VGtM = Volo's Guide to Monsters

• Night Hag (MM)
  
  
  • reproduce through the consumption of human children which later transform into hags
  
  


• Draegloth (VGtM)
  
  
  • half-drow, half-demon born in an unholy, dangerous ritual
  
  


• Barghest (VGtM)
  
  
  • born to goblin parents just like normal offspring and develops the ability to assume fiendish form
  
  


• Tanarukk (VGtM)
  
  
  • corrupt orcish bloodlines for generations upon breeding, bearing them in some female descendants
  
  

Using any of these creatures, you may be able to get some guidance into how a part-celestial, part-fiend would come about.

There is not a RAW version of this race in 5e. However, you can use the rules found in the Dungeon Master's Guide (see page 285) to create your own races and subraces.

None appear in 5th edition, though it is not intrinsically impossible

There is no creature in D&D 5th edition that represents the offspring of a fiend and a celestial. The offspring of a mortal and fiend or celestial are certainly possible, and earlier answers to this question cover the topic quite well.

However, in earlier editions of the game there was at least one instance of the offspring of two different types of fiends, specifically a demon and a devil. In D&D 3rd edition adventure Bastion of Broken Souls:

The Cathezar is one of Aameuls best agents. She is a fiend, but is an outcast among both demons and devils—she is half demon and half devil, spawn of a doubly unholy union.

The Cathezar has the abilities of both a kyton (chain devil) and marilith demon, suggesting that these were her parents. This suggests that canonically, creatures who are the offspring of two outsiders are possible, even among directly opposed enemies, although they would be extremely rare.

Another possibility is that someone might be the distant offspring of, say, an aasimar and a tiefling. There's no rule that says you can't be ⅛ aasimar and ⅛ tiefling; there's just no game statistics to represent that in D&D 5e, so you'd have to invent your own.

Nephalem (not RAW)

An offspring of a Celestial and a Demon is named a Nephalem, Nephalim or Nephilim in a lot of different universes.

Nephalem is a term heavily used by Blizzard in Diablo where they are human-free. From a Diablo fan wiki:

Nephalem are humanity's forebears. The offspring of angel and demon, their existence would forever alter the balance of power in the Great Conflict.

The Darksider franchise uses the Biblical term Nephilim. Per a Darksiders fan wiki:

Nephilim are an ancient and nearly extinct race born from a union of Angels and Demons engineered by the demoness Lilith when she created the first.

Devil May Cry also uses the Biblical term Nephilim. Per a Devil May Cry fan wiki:

The Nephilim are a race of hybrid beings within DmC: Devil May Cry continuity. They are the offspring of angels and demons; and are thus considered "the third race".

The term is common enough to be present in the homebrew part of D&D Beyond:

Aasimar are the descendants of celestials just as tieflings are to infernal beings. Nephalem are both. The result of the unholy union of Tiefling and Aasimar, Nephalem have both fiendish and celestial blood running through their veins. They are torn by a desire for good and justice, and resentment toward societies which shunned them. Many are rejected by one if not both parents, and forced to walk their path alone.

The Wikipedia article on Nephilim in popular culture has more information.