I've managed to figure out how to answer my own query. Having everyones input was incredibly helpful in teaching me about certain aspects of Python and how classes interact with dictionaries. All of your answers helped massively to guide me to my solution, so I am very grateful for the contributions from: "Neil Butcher", "Mark Tolonen" and "Marce Puente".

Here is how I managed to get the code to use the dictionary's values for each pokemon and also the data stored within the variables/instances of the class to find and then replace Pokemon's evolutions if they need updating.

Firstly, I needed to change the dictionary from having the keys for each variable as their pokedex number and just use their name as the key, to help the for loop I use later on with its comparisons.

pokemon_database = {
    "bulbasaur" : bulbasaur,
    "ivysaur" : ivysaur,
    "venusaur" : venusaur,
    "charmander" : charmander,
    "charmeleon" : charmeleon,
    "charizard" : charizard,
    "squirtle" : squirtle,
    "wartortle" : wartortle,
    "blastoise" : blastoise,
    "caterpie" : caterpie,
    "metapod" : metapod,
    "butterfree" : butterfree,
    "weedle" : weedle,
    "kakuna" : kakuna,
    "beedrill" : beedrill
    }

Then, after much trial and error with different versions of the for loop, I stumbled into creating this loop. As far as I can tell, it loops through the dictionary using the values of the stored items, rather than the keys, and compares these variables' stored "evolution" data if it is a string or not. If it is a string, then it replaces it with the corresponding variable name for the evolution it has found.

def update_evolutions(pokemon_database):
    for pkmon in pokemon_database.values():
        if pkmon.evolution:
            pkmon.evolution = pokemon_database[pkmon.evolution]
            updated_pokemon_list.append(pkmon.evolution.name) 

update_evolutions(pokemon_database)

Although not part of the actual loop, the "updated_pokemon_list" line is to add the Pokemon that have been found that needed updating, and then adding them to a list so I could check which Pokemon it had found during its loop that needed updating, just to check what it's doing.

I then added some code before and after the loop to make sure it was doing what I intended it to do, and I'll include that in its entirety for transparency.

# Debugging test - See which Pokemon were updateds
updated_pokemon_list: list = []

print(f"Bulbasaur evolves into {bulbasaur.evolution}, then it evolves eventually into {ivysaur.evolution}.") # Wouldn't let me add a further ".name" after each evolution since it caused an error. Showing it was still stored as a string.

# Automatically update any Pokemon's evolutions to link to the correct variable
def update_evolutions(pokemon_database):
    for pkmon in pokemon_database.values():
        if pkmon.evolution:
            pkmon.evolution = pokemon_database[pkmon.evolution]
            updated_pokemon_list.append(pkmon.evolution.name) 

update_evolutions(pokemon_database)


print(bulbasaur.evolution.name)
print(f"Bulbasaur evolves into {bulbasaur.evolution.name}, then it evolves eventually into {ivysaur.evolution.name}.")

print(updated_pokemon_list)

The output of this block was:

Bulbasaur evolves into ivysaur, then it evolves eventually into venusaur.

Ivysaur

Bulbasaur evolves into Ivysaur, then it evolves eventually into Venusaur.

['Ivysaur', 'Venusaur', 'Charmeleon', 'Charizard', 'Wartortle', 'Blastoise', 'Metapod', 'Butterfree', 'Kakuna', 'Beedrill']

Again, thank you for all your help.