You can keep things simple by splitting the queries:

cursor.execute('USE my_db')
cursor.execute('SELECT * FROM my_table')

GUI tools usually manage the context (i.e., the selected database) behind the scenes. Python connector is more strict—you need to explicitly fetch from each result in a multi=True call.