After a lot of discussions and serious chats with my hosting provider, I was unable to get a proper resolution from them, they keep saying it is out of their scope, and, it's the responsibility of my web designer to sort out this error. So I took it upon myself and got a temporary fix from Chat-GPT 4o. I'm posting the rule below for open critique. The following rule satisfies all my current need viz. Non-existent URLs ending with .php is giving 404 with custom Error page, manual 301 Redirects are working as intended, & there is no "file not found" error.

# 301 Redirects
Redirect 301 /old-page.php /new-page.php
 
### This is CHATGPT Magic
# Custom 404 for .php files
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_URI} \.php$ [NC]
RewriteRule ^.*$ /404.php [L,R=404]
 
# General 404 for non-.php URLs
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^.*$ /404.php [L,R=404]

Now only thing I really want is the explanation of why I was having file not found page rather than URL not found/custom 404 page. I think it is something related to PHP routing, but, as a non-expert, I'm not sure the exact cause.

Cheers!