big data summer 2017, revolution + firehose

hi all, have blogging up a storm for years now, coming up on the ½ decade anniversary. at times as far as topics its been like “kid in a candy store” for me. as the chinese say “may you live in interesting times”. there is so much going on these days, so many moving parts. we seem to be in the middle of major simultaneous revolutions in multiple fields, some of my favorites like CS+ physics. CS advancements in particular are rippling across many fields. CS is historically the field of infotech. but as noted in this blog, science is becoming increasingly about analysis of info/ data. a hybrid of CS+ statistical science is leading the way in analyzing Big Data. Big Data covers many areas and for years there was mostly a lot of focus on the hardware aspects. theres been huge innovation there and weve got awesome capabilities, systems, libraries for stuff like that, like nosql and graph databases. very impressive!

so now with those solid foundations in place, some of the applications/ advances/ breakthrus can commence. and we’re already seeing the “early” fruits all over the place. yet my feeling is that wrt the “long run” all of this is maybe only in 2nd gear.

have been tracking Big Data in my massive bookmark/ link piles. as you can imagine, thats a very difficult challenge, but am largely up to it with my hardcore cybersurfing and hacker tools eg bookmark export. and so a “huge” ~215 link list is just a drop in the bucket…!

made a conscious decision to hold off on writing up the links “for awhile” because AI has been taking center stage with deep learning and its largely merged with a lot of the cutting edge big data field. its getting increasingly hard to separate/ draw boundaries between them. but then, holy cow! THREE YEARS flew by in a flash. have been tracking a few big data related links in AI posts otherwise, but now my big data links runneth over and time to unleash/ release them. was somewhat waiting for a headline-grabbing event, but in a way they are happening constantly…

the sections are organized basic areas of innovation. its really cool to see it start to really percolate/ gel across multiple fields.

its hard to see the full themes of the 21st century at this early point, but there are several coalescing areas, and big data spread across multiple areas of STEM is a “big one”.

an interesting dichotomy that does emerge some, how does statistics tie in with big data? [c18][c27] are some blogs from a 175 year old statistics organization somewhat lamenting that Big Data is not fully connected to statistics in the publics mind and looking at outreach opportunities. quite ironic!

one notable area is ethics [b2] that is thankfully/ gratifyingly showing up significantly on the radar eg in books etc as people try to grapple with the human side of a massive emerging new technology. Big Data has very deep implications some of them verging/ impinging/ impacting on social, psychological, anthropological, cultural areas, etc…[d21] another big area was politics [b] with the recent election having massive big data implications. unf it seems these days like maybe russia “adversarily” mastered them better than we did in our own country. putin was recently quoted as asserting mass political power will come from AI but as with the ongoing fusion he may have been alluding the Big Data just as much.

so yeah, Big Data inherently has that firehose quality. how does one stay on top of it? its really like an entirely new crosscutting field and things are still “in the air” and the dust hasnt settled yet so to speak. we are living through a revolution materializing in front of our eyes. its a bit like/ reminiscent of the dotcom revolution starting out the early 21st century. however suspect this one could easily run hot for decades. yeah, all the way up to the singularity right? 😮 🙄

could write far, far more on this subject but just suggest you drill down into all the amazing material here where it interests you, theres some very high quality content in here and it rewards the time invested into it. am going to be poking on these links over the weeks/ months ahead and, to quote an old rock song about the birth of something awesome, “gaze in wide wonder at the joy they had found”

a. case study
a2. crime
a3. medical
a4. science
a5. business/ finance
a6. climate
a7. sports
a8. match
a9. writing
a10. recommend
a11. behavior
a12. prediction
a13. city
a14. competition
b. politics
b2. ethics
c. context/ overview
c2. future
c3. technique
d. rise/ fall
e. hype/ limits/ danger

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s