This is actually possible. Adding an option to your ChromeOptions called --headless=new doesn't open the simulator and you can get information from it,
here is a program which prints out the About tag in the google home page.
from selenium import webdriver
from selenium.webdriver.common.by import By
options = webdriver.ChromeOptions()
options.add_experimental_option('detach', True)
options.add_argument("--headless=new"). # This tells selenium to hide the browser
driver = webdriver.Chrome(options=options)
driver.get('https://www.google.com')
about = driver.find_element(By.XPATH, '/html/body/div[2]/div[2]/a[1]')
print(about)
You can also use requests with BeautifulSoup.
Here is how to set everything up:
from bs4 import BeautifulSoup
import requests
# Extract Website HTML
response = requests.get('https://www.google.com')
html = response.text
# Create BeautifulSoup Object
soup = BeautifulSoup(html, 'html.parser')
Now, to search for tags there is four main ways,
soup.select('css-selector') # Select all by css selector
soup.select_one('css-selector'). # Select one by css selector
and there is
soup.find().
soup.find_all()
You can add the attributes as a parameter and then the name as a value, for example, to get all tags with class being hello,
soup.find_all(class_='hello') # class_ since class exists already in python
or to get one tag with id 'link'
soup.find(id='link'). # Use find instead of find_all
Here is the BeautifulSoup docs:
https://www.crummy.com/software/BeautifulSoup/bs4/doc/
Also, look at this post: