Categories
Research

Q Posts Decoded Using Word Frequency Analysis

Using Python and the NLTK library (Natural Language Tool Kit) I analyzed all of Q’s posts up to May 16th 2020. I wanted to see which words were used the most often and which phrases come back more than once. I think you will find it quite interesting.

Here are the top 11 relevant words in Q’s posts:

Q (4735 times)
POTUS (513 times)
Think (487 times)
people (339)
about (326 times)
more (279 times)
been (214 times)
believe (203 times)
against (201 times)
FAKE (173 times)
NEWS (169 times)

And just for good measure, WWG1WGA!!! (99 times)

You probably noticed I’m not counting the stop words. Stop words are words such as I, you, is, can, be, etc. Stop words are generally ignored in natural language processing. I kept them intact for this exercise but I did not use them in the top 11 words. You’ll also notice that I kept the capital letters. Normally text is converted to all lowercase before doing analysis, (normalization) but because so much meaning is placed on little details such as capitalization, I left it intact. Let’s run the code again but this time we will include the line lower_text = data.lower() from the second part of the code below. Let’s see what it gives us.

q (4737 times)
have (648 times)
think (601 times)
potus (523 times)
people (456 times)
news (384 times)
public (292 times)
being (265 times)
fake (226 times)
believe (217 times)
against (216 times)

Now we’re getting somewhere. It’s clear from this analysis so far that the words POTUS and THINK are the most frequently used words.
In this next part, we will look at complete sentences or groups of words that re-appear in the text. We will use normalization for this analysis since we want to know the subject of the words. We won’t concern ourselves with capitalization in the next part and we’ll normalize the entire text to lower case first.

Some of the top 13 word phrases used in Q’s posts are:
What has been said about the US military? The speech yesterday verified and…. (and unlocked so much).
News in all forms unlocks the map. Expand your thinking. The great awakening.
for our struggle is not against flesh and blood but against the rulers…
the powers of this dark world and against the spiritual forces of evil…
be strong in the lord and his almighty power. Put on the full (armor of god)…

This is using groups of 13 words. As you can see below, these phrases keep coming back again and again in various forms.

Let’s look at a few more groups of words. First, let’s look at some of the top groups of 3 words together:
Why is this (158 times)
Is this relevant? (144 times)
Do you believe (139 times)
Believe in coincidences? (106 times)
These people are (104 times)
more than you (70 times)
have more than (57 times)
the great awakening (54 times)
the fake news (54 times)
future proves past (46 times)

and for good measure:
trust the plan (37 times)
fake news media (36 times)
not a game (35 times)
enjoy the show (24 times)

Let’s keep it going with groups of 4 words. Here are some of the top groups of 4 words:
Why is this relevant? (141 times)
Do you believe in (109 times)
you believe in coincidences (106 times)
more than you know (65 times)
have more than you (55 times)
of the united states (42 times)
these people are stupid (40 times)
this is not a (40 times)
not good enough – (impeach potus) (36 times)
enjoy the show. q (32 times)
president of the united (29 times)
the great awakening. q (29 times)
the fake news media (29 times)
full armor of god (20 times)

Ok. So where does that leave us? It leaves us with a general knowledge of what Q is trying to say (in my humble opinion). Here is the condensed version of the recurring themes:
1) POTUS and united states military. This is a recurring subject in many forms.
2) You have more than you know. Whatever this means, it seems important.
3) Do you believe in coincidences?
4) Put on the full armor of God
5) The fake news media
6) We have it all

These are fragments of text that keep showing up. I think the 13 word groupings analysis gives some good hints about how to proceed. It’s clear that yesterday’s speech is important. Q says a few times that you need to connect the markers and that news in all forms unlocks the map. It’s quite clear to me that these messages are the most important bits since they keep repeating. What does it mean exactly? I don’t know. I plan to analyze this further.

I hope you enjoyed this analysis. If you would like to preform your own analysis using Python and NLTK, I have provided the source code below.

Here is the python code to extract the text from the webpage:

import codecs
def remove_html_tags(text):
    """Remove html tags from a string"""
    import re
    clean = re.compile('<.*?>')
    return re.sub(clean, '', text)

from bs4 import BeautifulSoup
with open("posts.html", encoding="utf-8", errors="ignore") as f:
	data = f.read()
	soup = BeautifulSoup(data, 'html.parser')
	post_list = soup.findAll("div", {"class": "dont-break-out"})
	for post in post_list:
		cleantext=remove_html_tags(str(post))
		file1="qposts.txt" 
		text_file = codecs.open(file1,mode='a',encoding='utf-8', errors='ignore')
		text_file.write(cleantext)
		print(cleantext)

Once the text is extracted and separated from the HTML, you can use this code to do your own frequency analysis:

import nltk
with open ("qposts.txt", "r", encoding="utf-8", errors="ignore") as myfile:
    data=myfile.read().replace('\n', ' ')
lower_text = data.lower()
data = lower_text.split(' ')
fdist1 = nltk.FreqDist(data)
#Use the command below to print single words
#print (fdist1.most_common(200))
#The next section is used to analyze groups of words 
from nltk import ngrams, FreqDist
all_counts = dict(fdist1)
for size in 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13:
    all_counts[size] = FreqDist(ngrams(data, size))
#Change the number 13 to any number from the list above
print(all_counts[13].most_common(100))

Show your support for my work by following me on social media or by sending me cryptocurrency:

Bitcoin: 3MAt2kKf8jfDgc5NgBckwQZN5HCB1P78T3
Ethereum: 0xA9F35a279F4565b413a860746f20c5B3d248a923

Facebook: https://www.facebook.com/DanJos3ph
Twitter: https://twitter.com/djos3ph
Instagram: https://instagram.com/d.j.o.s.e.p.h
Youtube: https://www.youtube.com/channel/UCdGk1hXpJ7OAQU3pcz05UGA

Thank you! Where we go one, we go all!
– Dan Joseph

Leave a Reply