Tweets and Stock Price Movement Analysis using Postgres and RDS (AWS)

A solo project built to see whether a PostgreSQL-on-AWS-RDS pipeline could quantify the relationship between cashtag-tagged tweet sentiment and the underlying asset's price movement — taking a single high-volume cashtag ($ETH) and riding it end-to-end from raw tweets to a materialised-view mini-warehouse to a dual-axis chart.

Tech: Python (pandas, TextBlob, SQLAlchemy, psycopg2) · PostgreSQL · AWS RDS · pgAdmin · Jupyter

Why tweets → stocks

The academic inspiration here is Bollen, Mao & Zeng's 2011 paper "Twitter mood predicts the stock market," which famously showed that aggregate Twitter sentiment had some predictive power on Dow Jones closing values a few days later. That paper was a big deal when it came out because it gave a concrete, testable version of the "wisdom of crowds" hypothesis — that the distributed emotional state of millions of people, each of whom knows a little about something, might contain information that the market hasn't priced in yet.

I wanted to see how much of that claim survives contact with a more modest setup: a single cashtag, a single asset, a single year, and off-the-shelf sentiment scoring. The actual question: at the scale you can build in a side project, does tweet sentiment track price movement at all — and if it does, does it lead, lag, or move together?

The answer (spoiler) turned out to be "weakly, directionally, and mostly in a way that would not make you money" — which is a useful result, because it's the result most such pipelines produce, and it's the one papers don't tend to publish.

I picked $ETH because the cashtag dataset I pulled (Zenodo DOI 10.5281/zenodo.2686861) was heavily weighted toward it. I considered the obvious candidates (TSLA, AMZN, meme names) but the dataset's actual density was in crypto-adjacent cashtags, and rather than fight the data I followed where it was thickest. $ETH in 2017 has the additional nice property of a real run-up and a real drawdown inside the window, so you're not just staring at a flat line trying to find a 2% correlation.

The wisdom-of-crowds hypothesis

A short detour on what the hypothesis is actually claiming. Twitter sentiment could be informative about future prices for a few different reasons:

Genuine information. A subset of tweet authors actually know something — they're traders, they're close to a company, they spotted a filing — and their tweets encode that information before it's reflected in the price.
Collective mood. Even if no individual tweet is informative, the aggregate sentiment captures some affective state that correlates with buying or selling pressure across the broader market.
Coordination. Pump-and-dump campaigns, coordinated FUD, influencer calls. Tweets aren't describing the market — they're trying to move it.
Nothing. Tweets lag the price, because people tweet about what just happened. Or the relationship is so noisy that it's dominated by unrelated external events on almost every day.

All four of those can be true simultaneously, and for a cashtag like $ETH in 2017, I'd bet all four were. That's what makes "correlation between sentiment and price" a weak test: the same correlation is consistent with very different generative stories, and you can't distinguish them from the time series alone.

Architecture

flowchart LR
  A[Zenodo<br/>finance-tagged tweets] --> C[Python ingest<br/>psycopg2 COPY]
  B[Yahoo Finance + NYSE<br/>symbol directory] --> C
  C --> D[(PostgreSQL<br/>on AWS RDS)]
  D --> E[TextBlob<br/>sentiment scoring]
  E --> D
  D --> F[Materialized view<br/>stock_sentiment_analysis<br/>+ B-tree indexes]
  F --> G[Daily aggregation<br/>SQL]
  G --> H[Matplotlib<br/>dual-axis chart]

The design decision I'm most proud of here is keeping the heavy joining and aggregation in Postgres rather than dragging everything into a notebook. Pandas is a wonderful tool for 100k rows; it is a bad tool for 6 million rows of tweets joined against a symbol directory joined against a year of daily price data. SQL was designed for that workload. Once I committed to "the database is the computation layer," everything downstream got easier.

Data choices

Tweets. The Zenodo cashtag dataset is a collection of finance-tagged tweets (cashtag format, like $AAPL or $ETH) scraped over 2017. It's not exhaustive — it samples — but it's cleanly structured, has the tweet IDs, timestamps, cashtag extractions already done, and crucially it's publicly cite-able so my work is reproducible by anyone who can download the same DOI.