Trabajando en AWS Cloud9, mi script es:

from bs4 import BeautifulSoup
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
import requests


options = Options()
options.binary_location = '/usr/bin/google-chrome'
driver = webdriver.Chrome(chrome_options=options, executable_path='/home/ec2-user/environment/downloads/chromedriver')
url = 'http://google.com'
driver.get(url)

soup = BeautifulSoup(driver.page_source, 'lxml')

# driver.quit()    

print(soup)

Sin embargo, cuando ejecuto el archivo, la configuración de ejecución se cuelga hasta que el teclado interrumpe con ctrl + c. Traceback devuelve esto:

> Traceback (most recent call last):   File
> "/home/ec2-user/environment/vpy36/lib/python3.6/dist-packages/hubspot-marketplace.py",
> line 9, in <module>
>     driver = webdriver.Chrome(chrome_options=options, executable_path='/home/ec2-user/environment/downloads/chromedriver')  
> File
> "/home/ec2-user/environment/vpy36/lib/python3.6/dist-packages/selenium/webdriver/chrome/webdriver.py",
> line 75, in __init__
>     desired_capabilities=desired_capabilities)   File "/home/ec2-user/environment/vpy36/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py",
> line 154, in __init__
>     self.start_session(desired_capabilities, browser_profile)   File "/home/ec2-user/environment/vpy36/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py",
> line 243, in start_session
>     response = self.execute(Command.NEW_SESSION, parameters)   File "/home/ec2-user/environment/vpy36/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py",
> line 310, in execute
>     response = self.command_executor.execute(driver_command, params)   File
> "/home/ec2-user/environment/vpy36/lib/python3.6/dist-packages/selenium/webdriver/remote/remote_connection.py",
> line 466, in execute
>     return self._request(command_info[0], url, body=data)   File "/home/ec2-user/environment/vpy36/lib/python3.6/dist-packages/selenium/webdriver/remote/remote_connection.py",
> line 490, in _request
>     resp = self._conn.getresponse()   File "/usr/lib64/python3.6/http/client.py", line 1331, in getresponse
>     response.begin()   File "/usr/lib64/python3.6/http/client.py", line 297, in begin
>     version, status, reason = self._read_status()   File "/usr/lib64/python3.6/http/client.py", line 258, in _read_status
>     line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")   File "/usr/lib64/python3.6/socket.py", line 586, in readinto
>     return self._sock.recv_into(b) KeyboardInterrupt

Solo pasé las últimas 4 horas tratando de instalar Chrome y BeautifulSoup en AWS Cloud 9. Muy difícil.

Edit: simplemente lo dejé correr hasta que se detuvo por sí solo y devolvióMessage: unknown error: Chrome failed to start: exited abnormally

Edit 2: Información del controlador: chromedriver = 2.10.267518,

plataforma = Linux 4.9.85-38.58.amzn1.x86_64 x86_64

Google Chrome 66.0.3359.117