Use the MetafieldSet
I am unable to paste into here, so I will try and type what you need (may have some typos)

mutation MetafieldsSet($metafields: [MetafieldSetInput!]!){
metafieldSet(metafields: $metafields){
metafields
{
id
namespace
key
value
}
userErrors {
field
message
elementIndex
}
}
}

Your upsert variables should be along the lines of the following:

"metafields" : [ {
"key" : "color-pattern",
"namespace" :"shopify",
"ownerId": "gid://shopify/Product/<PRODNUMBER>",
"type": "list.metaobject_reference",
"value": "[\"gid://shopify/Metaobject/<META OBJ ID>\"]"
}]

to find the specific meta object id, I like to use the browser dev tools, open up your product page in shopify, select the category meta field properties you want to add, and before saving

go to network tab,
click the clear button to remove any resources shown
filter by : type=mutation

for more filters click on Fetch/XHR

go ahead and save,

in the network tab list on the left you will see a URL name in the list
<storeID>?operation=MetafiedsSet&type=mutation

if you select it, you can then view the payload to see what variables shopify is setting in the admin UI