I got it working, I think the example in the link above is old. The below code worked for me and I was able to create a prompt programmatically and see in vertex AI studio. Still trying to see how to manage version and compare prompts. Also it looks to me that to use generative ai on GCP, we will need both the vertexai and the google-genai package. It looks like generative AI models are removed from vertexai and moved to google-genai. If I am wrong on this, would like to be be corrected.

I got the below code here https://github.com/googleapis/python-aiplatform

import vertexai

# Instantiate GenAI client from Vertex SDK
# Replace with your project ID and location
client = vertexai.Client(project='xxx', location='us-central1')

prompt = {
    "prompt_data": {
        "contents": [{"parts": [{"text": "Hello, {name}! How are you?"}]}],
        "system_instruction": {"parts": [{"text": "Please answer in a short sentence."}]},
        "variables": [
            {"name": {"text": "Alice"}},
        ],
        "model": "gemini-2.5-flash",
    },
}

prompt_resource = client.prompts.create(
    prompt=prompt,
)

print(prompt_resource)