SQLAlchemy 2.0 Breaks RSS #239

New issue

Closed

opened 2023-02-22 10:07:51 +01:00 by joshblake · 2 comments

joshblake commented

2023-02-22 10:07:51 +01:00

It seems that SQLAlchemy installed via pip breaks the RSS plugin with the following stack trace:

2023-02-22 09:21:11,007| INFO|platypush|---- Starting platypush v.0.24.4
2023-02-22 09:21:12,310| INFO|platypush:backend:http|Initializing websocket interface
2023-02-22 09:21:12,315| INFO|websockets.server|server listening on 0.0.0.0:8009
2023-02-22 09:21:12,336| INFO|platypush:backend:http|Starting local web server on port 8008
2023-02-22 09:21:12,342| INFO|platypush| * Serving Flask app 'platypush'
2023-02-22 09:21:12,342| INFO|platypush| * Debug mode: off
2023-02-22 09:21:12,346| INFO|werkzeug|WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on all addresses (0.0.0.0)
 * Running on http://127.0.0.1:8008
 * Running on http://192.168.1.3:8008
2023-02-22 09:21:12,347| INFO|werkzeug|Press CTRL+C to quit
2023-02-22 09:21:12,729| INFO|platypush:plugin:rss|Parsing feed subscriptions
2023-02-22 09:21:19,424|WARNING|utils|Could not initialize plugin rss
2023-02-22 09:21:19,425|ERROR|utils|Not an executable object: 'CREATE TABLE IF NOT EXISTS variable(\n            name varchar(255) not null primary key,\n            value text\n        )'
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/engine/base.py", line 1410, in execute
    meth = statement._execute_on_connection
AttributeError: 'str' object has no attribute '_execute_on_connection'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/platypush/utils/__init__.py", line 433, in get_enabled_plugins
    plugin = get_plugin(name)
  File "/usr/local/lib/python3.9/dist-packages/platypush/context/__init__.py", line 131, in get_plugin
    plugins[plugin_name] = plugin_class(**plugin_conf)
  File "/usr/local/lib/python3.9/dist-packages/platypush/plugins/rss/__init__.py", line 65, in __init__
    self._latest_timestamps = self._get_latest_timestamps()
  File "/usr/local/lib/python3.9/dist-packages/platypush/plugins/rss/__init__.py", line 83, in _get_latest_timestamps
    return {url: self._get_feed_latest_timestamp(url) for url in self.subscriptions}
  File "/usr/local/lib/python3.9/dist-packages/platypush/plugins/rss/__init__.py", line 83, in <dictcomp>
    return {url: self._get_feed_latest_timestamp(url) for url in self.subscriptions}
  File "/usr/local/lib/python3.9/dist-packages/platypush/plugins/rss/__init__.py", line 74, in _get_feed_latest_timestamp
    get_plugin('variable')
  File "/usr/local/lib/python3.9/dist-packages/platypush/context/__init__.py", line 131, in get_plugin
    plugins[plugin_name] = plugin_class(**plugin_conf)
  File "/usr/local/lib/python3.9/dist-packages/platypush/plugins/variable/__init__.py", line 34, in __init__
    self._create_tables()
  File "/usr/local/lib/python3.9/dist-packages/platypush/plugins/variable/__init__.py", line 38, in _create_tables
    self.db_plugin.execute("""CREATE TABLE IF NOT EXISTS {}(
  File "/usr/local/lib/python3.9/dist-packages/platypush/plugins/__init__.py", line 21, in _execute_action
    result = f(*args, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/platypush/plugins/db/__init__.py", line 93, in execute
    connection.execute(statement)
  File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/engine/base.py", line 1412, in execute
    raise exc.ObjectNotExecutableError(statement) from err
sqlalchemy.exc.ObjectNotExecutableError: Not an executable object: 'CREATE TABLE IF NOT EXISTS variable(\n            name varchar(255) not null primary key,\n            value text\n        )'

The service will load if RSS is not enabled. Installing SQLAlchemy < 2.0 seems to fix it temporarily.

It seems that SQLAlchemy installed via pip breaks the RSS plugin with the following stack trace: ``` 2023-02-22 09:21:11,007| INFO|platypush|---- Starting platypush v.0.24.4 2023-02-22 09:21:12,310| INFO|platypush:backend:http|Initializing websocket interface 2023-02-22 09:21:12,315| INFO|websockets.server|server listening on 0.0.0.0:8009 2023-02-22 09:21:12,336| INFO|platypush:backend:http|Starting local web server on port 8008 2023-02-22 09:21:12,342| INFO|platypush| * Serving Flask app 'platypush' 2023-02-22 09:21:12,342| INFO|platypush| * Debug mode: off 2023-02-22 09:21:12,346| INFO|werkzeug|WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead. * Running on all addresses (0.0.0.0) * Running on http://127.0.0.1:8008 * Running on http://192.168.1.3:8008 2023-02-22 09:21:12,347| INFO|werkzeug|Press CTRL+C to quit 2023-02-22 09:21:12,729| INFO|platypush:plugin:rss|Parsing feed subscriptions 2023-02-22 09:21:19,424|WARNING|utils|Could not initialize plugin rss 2023-02-22 09:21:19,425|ERROR|utils|Not an executable object: 'CREATE TABLE IF NOT EXISTS variable(\n name varchar(255) not null primary key,\n value text\n )' Traceback (most recent call last): File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/engine/base.py", line 1410, in execute meth = statement._execute_on_connection AttributeError: 'str' object has no attribute '_execute_on_connection' The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.9/dist-packages/platypush/utils/__init__.py", line 433, in get_enabled_plugins plugin = get_plugin(name) File "/usr/local/lib/python3.9/dist-packages/platypush/context/__init__.py", line 131, in get_plugin plugins[plugin_name] = plugin_class(**plugin_conf) File "/usr/local/lib/python3.9/dist-packages/platypush/plugins/rss/__init__.py", line 65, in __init__ self._latest_timestamps = self._get_latest_timestamps() File "/usr/local/lib/python3.9/dist-packages/platypush/plugins/rss/__init__.py", line 83, in _get_latest_timestamps return {url: self._get_feed_latest_timestamp(url) for url in self.subscriptions} File "/usr/local/lib/python3.9/dist-packages/platypush/plugins/rss/__init__.py", line 83, in <dictcomp> return {url: self._get_feed_latest_timestamp(url) for url in self.subscriptions} File "/usr/local/lib/python3.9/dist-packages/platypush/plugins/rss/__init__.py", line 74, in _get_feed_latest_timestamp get_plugin('variable') File "/usr/local/lib/python3.9/dist-packages/platypush/context/__init__.py", line 131, in get_plugin plugins[plugin_name] = plugin_class(**plugin_conf) File "/usr/local/lib/python3.9/dist-packages/platypush/plugins/variable/__init__.py", line 34, in __init__ self._create_tables() File "/usr/local/lib/python3.9/dist-packages/platypush/plugins/variable/__init__.py", line 38, in _create_tables self.db_plugin.execute("""CREATE TABLE IF NOT EXISTS {}( File "/usr/local/lib/python3.9/dist-packages/platypush/plugins/__init__.py", line 21, in _execute_action result = f(*args, **kwargs) File "/usr/local/lib/python3.9/dist-packages/platypush/plugins/db/__init__.py", line 93, in execute connection.execute(statement) File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/engine/base.py", line 1412, in execute raise exc.ObjectNotExecutableError(statement) from err sqlalchemy.exc.ObjectNotExecutableError: Not an executable object: 'CREATE TABLE IF NOT EXISTS variable(\n name varchar(255) not null primary key,\n value text\n )' ``` The service will load if RSS is not enabled. Installing SQLAlchemy < 2.0 seems to fix it temporarily.

👀 1

blacklight commented

2023-02-22 11:16:39 +01:00

Owner

Hi @joshblake, thanks for reporting the error.

This is indeed a regression due to the latest major release of SQLAlchemy (released 3 weeks ago), which introduced many breaking changes. Most of them are actually a good thing though, support for raw SQL statements has been limited and that will prompt me to clean up some parts of the codebase that I was planning to clean up anyway.

Installing the latest version of Platypush (released yesterday) should fix the issue, as it requires sqlalchemy < 2.0.0 in the setup.py.

However, I should definitely plan to migrate the codebase to SQLAlchemy 2.x - all the major distros still ship SQLAlchemy 1.x, but that will obviously change in the upcoming weeks/months, and Platypush' dependencies should never conflict with system dependencies.

I will work on a PR to upgrade the compatibility with the latest version soon.

Hi @joshblake, thanks for reporting the error. This is indeed a regression due to the latest major release of SQLAlchemy (released 3 weeks ago), which introduced many breaking changes. Most of them are actually a good thing though, support for raw SQL statements has been limited and that will prompt me to clean up some parts of the codebase that I was planning to clean up anyway. Installing the latest version of Platypush ([released yesterday](https://pypi.org/project/platypush/0.24.5/)) should fix the issue, as it requires `sqlalchemy < 2.0.0` in the `setup.py`. However, I should definitely plan to migrate the codebase to SQLAlchemy 2.x - all the major distros still ship SQLAlchemy 1.x, but that will obviously change in the upcoming weeks/months, and Platypush' dependencies should never conflict with system dependencies. I will work on a PR to upgrade the compatibility with the latest version soon.

blacklight added the

bug

label 2023-02-22 11:18:08 +01:00

blacklight self-assigned this 2023-02-22 11:18:13 +01:00

blacklight added this to the core project 2023-02-22 11:18:21 +01:00

blacklight added this to the SQLAlchemy 2.x migration milestone 2023-02-22 11:19:42 +01:00

blacklight referenced this issue from a pull request that will close it,

2023-04-24 22:54:05 +02:00

Fixed compatibility with SQLAlchemy >= 2.0 #250

blacklight closed this issue

2023-04-25 10:47:28 +02:00

blacklight referenced this issue from a commit

2023-04-25 10:47:28 +02:00

Merge pull request 'Fixed compatibility with SQLAlchemy >= 2.0' (#250) from 239-sqlalchemy-2-compatibility into master

blacklight commented

2023-04-25 10:57:37 +02:00

Owner

This is now fixed on master, I've tested Platypush with SQLAlchemy versions from 1.3 and 2.0 (which should cover anything currently installed on any major distros + the future specifications) and things work.

You can install the new version via git, or wait for the next (big) pip release (which should come up in a few days at most).

This is now fixed on `master`, I've tested Platypush with SQLAlchemy versions from 1.3 and 2.0 (which should cover anything currently installed on any major distros + the future specifications) and things work. You can install the new version via git, or wait for the next (big) pip release (which should come up in a few days at most).