The reason for the restriction is related to variance. Task is a class, and classes in C# are ALWAYS invariant. The compiler expects that your Task can be substituted with some type, but the substitution is not permitted since Task is invariant - hence the issue.
There is not a workaround to having Task as a covariant type parameter since it is invariant by definition.