You can knock a creature out with a melee attack

Knocking a Creature Out

Sometimes an attacker wants to incapacitate a foe, rather than deal a
killing blow. When an attacker reduces a creature to 0 hit points with
a melee attack, the attacker can knock the creature out. The attacker
can make this choice the instant the damage is dealt. The creature
falls unconscious and is stable. (basic rules)

Note that this only works with melee attacks. So ranged attacks will still kill. Though it is good to note that melee spell attacks do still count under the rule (see another of my answers for a bit more discussion here).

This is the only rule that provides a mechanic for knocking out an opponent with an attack instead of doing lethal damage.

1. 
   

   Knock a Creature Out (Player's Handbook, page 198)

   
   
   

   
   

   Sometimes an attacker wants to incapacitate a foe,
   rather than deal a killing blow. When an attacker
   reduces a creature to O hit points with a melee attack,
   the attacker can knock the creature out. The attacker
   can make this choice the instant the damage is dealt.
   The creature falls unconscious and is stable.

   
   

   
   


2. 
   

   Disarm (Action Options, Dungeon Master's Guide, page 271)

   
   
   

   
   

   A creature can use a weapon attack to knock a weapon
   or another item from a target's grasp. The attacker
   makes an attack roll contested by the target's Strength
   (Athletics) check or Dexterity (Acrobatics) check. If the
   attacker wins the contest, the attack causes no damage
   or other ill effect, but the defender drops the item.

   
   
   

   The attacker has disadvantage on its attack roll if
   the target is holding the item with two or more hands.
   The target has advantage on its ability check if it is
   larger than the attacking creature, or disadvantage if it
   is smaller.

   
   

   
   


3. 
   

   Grapple (Player's Handbook, page 195)

   
   
   

   
   

   If you succeed, you subject the target to
   the grappled condition (see appendix A). The condition
   specifies the things that end it, and you can release the
   target whenever you like (no action required).

   
   

   
   


4. 
   

   Use the Grappler feat (Player's Handbook, page 167) to restrain the target. Please note that this actually imposes the restrained condition, not the grappled condition.

   
   
   

   
   

   You have advantage on attack rolls against a creature
   you are grappling.
   You can use your action to try to pin a creature grappled by you. To do so, make another grapple check. If
   you succeed, you and the creature are both restrained
   until the grapple ends.

   
   

   
   


5. 
   

   Shoving (Player's Handbook, page 195)

   
   
   

   
   

   Using the Attack action, you can make a special melee
   attack to shove a creature, either to knock it prone or
   push it away from you. If you're able to make multiple
   attacks with the Attack action, this attack replaces
   one of them.

   
   

   
   

Demand Surrender

It's not a hard and fast rule, like grappling or knocking a creature out, but you can always demand an enemy in a losing situation surrender in exchange for their life. If your are handily wiping the floor with some bandits, they're likely to accept, barring some sort of code of honor.

Besides, there's practically no cost to trying; in the worst case, they keep fighting a losing battle, and you use the more direct "knock them out" option. Though really, why risk a continued fight, when you could end it here and now with a polite request?

Listen up, kitty cat. Unless you lay down your weapons in the next 5 seconds, Dellan here is going to blast you with fire so hot, you won't be able to tell yourself from your campfire. Drop that sword, and we can settle this more cool headedly.

Disclaimer

Unlike the other answer, whether this attempt works outright, fails outright, or requires an ability check is up to the DM. Since speaking is a free action, it's likely a demand for surrender wouldn't take an action, but there's no specific rule, so again it's up to the DM.

Aside from the ways mentioned here, there are 3 more unconventional options on pages 272 and 273 of the Dungeon Master’s Guide, those being Lingering Injuries, Massive Damage and Morale.

Starting with Lingering Injuries:

It's up to you to decide when to check for a lingering injury. A creature might sustain a lingering injury under the following circumstances:
When it takes a critical hit,
When it drops to 0 hit points but isn't killed outright, When it fails a death saving throw by 5 or more

The injuries most likely to interest you are:

‘lose a hand/arm’ where a creature can no longer hold weapons with the ‘two handed’ property and can only hold one object. If you cut off a hand wielding a weapon, you could rule the creature is Disarmed.

‘Lose a foot/leg’ where movement speed is halved, the creature has disadvantage on checks to balance and falls prone after using the Dash action. You could rule if you cut off a leg, the creature falls prone.

And ‘lose an eye’ where a creature has disadvantage on ranged attacks and Wisdom (perception) checks. If you have no eyes left, you are Blinded. You could rule a creature who loses an eye becomes Stunned.

Moving on, Massive Damage occurs when a creature takes damage from a single source equal to or greater than half their hit point maximum. A creature must succeed a DC 15 Constitution saving throw or roll on the System Shock table. Effects range from falling to 0 Hit Points to being stunned to losing reactions until the end of the turn.

Finally, Morale. This is an optional rule where a creature, or group of creatures, may choose to flee combat rather than fight. If a creature loses half its HP maximum for the first time in combat, is surprised or has no way of harming its opponent, it may flee. If a group is surprised, have no way of harming an opponent, are reduced to half their original numbers or their leader is killed or incapacitated, the group may flee.

In order to determine if a creature or group flees, the creature or leader must pass a DC 10 Charisma check. You can also rule that the creature or group automatically fails. On a successful save, the creature or group remains in the fight. On a fail, the group flees and, if they cannot flee, they surrender. If they are then attacked, they will continue to fight.

All of these can be used to neutralise or disable a threat (some more literally than others) without outright killing them.