You dont need to think about direction, instead of current floor of the elevator you should chose the destination floor.
The order shoud be unocupied > ocupied
if both unocupied choose current.closer
if both at the same floor choose elevator 1 (just a arbitrary because it wont matter)
if both occupied choose destination.closer
if elv1.destination = elv2.destination choose 1.

you could use a counter, once goes elv1, then elv2, then 1, then 2. so both elevators get used more evenly
this logic also lacks if 2 or more people press the button at the same time... anyway good luck