1073
Hot data science related posts every hour. Chat: https://telegram.me/r_channels
R Greg Yang's work on a rigorous mathematical theory for neural networks
Greg Yang is a mathematician and AI researcher at Microsoft Research who for the past several years has done incredibly original theoretical work in the understanding of large artificial neural networks. His work currently spans the following five papers:
Tensor Programs I: Wide Feedforward or Recurrent Neural Networks of Any Architecture are Gaussian Processes: https://arxiv.org/abs/1910.12478
Tensor Programs II: Neural Tangent Kernel for Any Architecture: https://arxiv.org/abs/2006.14548
Tensor Programs III: Neural Matrix Laws: https://arxiv.org/abs/2009.10685
Tensor Programs IV: Feature Learning in Infinite-Width Neural Networks: https://proceedings.mlr.press/v139/yang21c.html
Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer: https://arxiv.org/abs/2203.03466
In our whiteboard conversation, we get a sample of Greg's work, which goes under the name "Tensor Programs". The route chosen to compress Tensor Programs into the scope of a conversational video is to place its main concepts under the umbrella of one larger, central, and time-tested idea: that of taking a large N limit. This occurs most famously in the Law of Large Numbers and the Central Limit Theorem, which then play a fundamental role in the branch of mathematics known as Random Matrix Theory (RMT). We review this foundational material and then show how Tensor Programs (TP) generalizes this classical work, offering new proofs of RMT.
We conclude with the applications of Tensor Programs to a (rare!) rigorous theory of neural networks. This includes applications to a rigorous proof for the existence of the Neural Network Gaussian Process and Neural Tangent Kernel for a general class of architectures, the existence of infinite-width feature learning limits, and the muP parameterization enabling hyperparameter transfer from smaller to larger networks.
​
https://preview.redd.it/av3ovotcunaa1.png?width=1280&format=png&auto=webp&s=dae42e6b7c41a15acd6b5eeb752b8db064d3e8da
https://preview.redd.it/hh9q6wqdunaa1.png?width=1200&format=png&auto=webp&s=b2936e129d9444fc5434a4c3f5b36315d3e06057
Youtube: https://youtu.be/1aXOXHA7Jcw
Apple Podcasts: https://podcasts.apple.com/us/podcast/the-cartesian-cafe/id1637353704
Spotify: https://open.spotify.com/show/1X5asAByNhNr996ZsGGICG
RSS: https://feed.podbean.com/cartesiancafe/feed.xml
/r/MachineLearning
https://redd.it/105v7el
9 Exploding eCommerce Niches in 2023
/r/Infographics
https://redd.it/1038g3l
Resource for interesting data science project notebooks
I am an experienced Senior Data Scientist looking for repositories with interesting, well-curated Data Science projects in Jupyter notebook format.
Want to spend every day 45 minutes with code examples, spanning different challenging and interesting topics.
Most books have boring topics.
Can you recommend something? Kaggle might be it, but I don’t know how to find well-curated solutions there.
/r/datascience
https://redd.it/105qppp
[OC] 2 Decades of Improving Racial Acceptance: White Americans Are Increasingly Open to a Close Relative Marrying Any Race
/r/dataisbeautiful
https://redd.it/105oucb
Volume of Dead Crypto Coins by death year/ start year
https://redd.it/1032m22
@datascientology
R The Evolutionary Computation Methods No One Should Use
So, I have recently found that there is a serious issue with benchmarking evolutionary computation (EC) methods. The ''standard'' benchmark set used for their evaluation has many functions that have the optimum at the center of the feasible set, and there are EC methods that exploit this feature to appear competitive. I managed to publish a paper showing the problem and identified 7 methods that have this problem:
https://www.nature.com/articles/s42256-022-00579-0
Now, I performed additional analysis on a much bigger set of EC methods (90 considered), and have found that the center-bias issue is extremely prevalent (47 confirmed, most of them in the last 5 years):
https://arxiv.org/abs/2301.01984
Maybe some of you will find it useful when trying out EC methods for black-box problems (IMHO they are still the best tools available for such problems).
/r/MachineLearning
https://redd.it/1051j8j
The Longest Lasting Cars, in Miles
/r/Infographics
https://redd.it/1032rok
Floods cutting roads in northwestern Australia has turned a ~4600km return trip into a ~12000km return trip.
/r/MapPorn
https://redd.it/1057kdo
I just finished this map doily pattern and thought you all would enjoy seeing it
/r/MapPorn
https://redd.it/104ykg3
TopicOpen Open Discussion Thread — Anybody can post a general visualization question or start a fresh discussion!
Anybody can post a question related to data visualization or discussion in the monthly topical threads. Meta questions are fine too, but if you want a more direct line to the mods, click here
If you have a general question you need answered, or a discussion you'd like to start, feel free to make a top-level comment.
Beginners are encouraged to ask basic questions, so please be patient responding to people who might not know as much as yourself.
---
To view all Open Discussion threads, click here.
To view all topical threads, click here.
Want to suggest a topic? Click here.
/r/dataisbeautiful
https://redd.it/100j6br
% of gay or lesbian people within England and Wales.
/r/MapPorn
https://redd.it/1058fc4
Map of broad U.S regions
/r/MapPorn
https://redd.it/104ypxz
[D] Fixing the angle of Skewed Paintings, see comments
https://redd.it/104u2di
@datascientology
The many types of beers and their countries of origin
/r/Infographics
https://redd.it/103b1u9
The most expensive buildings ever constructed
/r/Infographics
https://redd.it/104499i
[OC] Biggest Flower Exporters
/r/dataisbeautiful
https://redd.it/105ts8l
[OC] Map showing temperature anomalies over the northern hemispher on New Year's Day
/r/dataisbeautiful
https://redd.it/105olnr
[OC] Metal bands bring happiness (as chocolate brings Nobel Prizes)
/r/dataisbeautiful
https://redd.it/105qgoi
2022 in scientific headlines
/r/visualization
https://redd.it/105ov2l
Why are there more remote positions in the US than in EU
I am trying to get a remote position as a data scientist in EU but it seems like there are not many opportunities. Meanwhile when I change the location to the US there are about 100 times more position. I am wondering what the reason could be?
/r/datascience
https://redd.it/1052dli
How Stable was Each Country in 2022? (According to Fragile State Index)
/r/MapPorn
https://redd.it/105gi5x
"Everyone You Will Ever Meet Knows Something You Don't" - The 9 Types of Intelligence
/r/Infographics
https://redd.it/105h3kt
The Ultimate Cheatsheet For Kritical Thinking
/r/Infographics
https://redd.it/105gxbv
3D Population Density Map of Japan [OC] (Data source: Worldpop.org / Software: QGIS and Blender)
/r/dataisbeautiful
https://redd.it/105d400
[OC] Cars make suburbs riskier than cities
/r/dataisbeautiful
https://redd.it/104vty9
The Most Important Science Headlines of 2022
/r/Infographics
https://redd.it/104yyps
Serb and Italian majority areas in Croatia (1931 - 2011) [OC]
/r/MapPorn
https://redd.it/104tv1i
The last country each country was ruled by.
/r/MapPorn
https://redd.it/104vzjl
Q Book suggestion for Spatial Statistics / Geostatistics
Hello everyone.
I am in desperate need of book suggestion for spatial statistics (I would say Geostatistics). Let me explain my situation: The books are either too easy, or too hard. I skimmed/studied at least 10 books so far and none of them works. For example, “Applied Geostatistics” by Isaaks and Srivastava is a very good book with good explanation, however, it is way too lightweight and only "talks about" concepts. On the other hand, "Geostatistics: Modeling Spatial Uncertainty" by Chiles and Delfiner has all the topics I want to study, but it starts to delve into crazy concepts even from Chapter 1, without explaining what is what in the first place. Some books talk about Variograms on Chapter 2, and some other books talk about them on Chapter 22 (literally, check Y.Z. Ma's book).
In essence, I need to learn about Variograms, Kriging and Simulations (I am OK with Unconditionals at the beginning). If I need to read a prerequisite book, I am also OK with that. If I need to take a prerequisite lecture, I am also OK with that. Please guide me dear Redditors. I have studied many topics in my life but none of them felt this arcane*.* It feels like I am trying to learn some occult. It is never explained clearly, it is available nowhere and it is always super-complicated or it is extremely dumbed-down to the point it is not what it is anymore. If there is a book, even 5000 pages long, that has the topics of Chiles&Delfiner, but the explanation of Isaaks & Srivastava, please tell me the name.
/r/statistics
https://redd.it/104lt89
What’s your favourite swear word? (Everyone 13+)
Anyone can take this survey, as all of the demographic questions are optional and most include an “other” option to be inclusive of options that I may have not thought of.
https://docs.google.com/forms/d/e/1FAIpQLSfp8kTuUfFBQ1funASmvQ-jsJh8qpb4erOluQgGWBNYh2sgw/viewform?usp=sflink
/r/SampleSize
https://redd.it/1046vud
