68
Dev and IT related stuff. Rate us here: https://telegram.me/tchannelsbot?start=x4516 Feedback: https://t.me/ox49d1
Wow, appears that Marvel has some serious developer related sh8t! The thing is not new, but still…
They have an API that can be used to make some interactive programs, related to their comics' series, with amazingly prepared docs pages.
The Marvel Comics API is a RESTful service which provides methods for accessing specific resources at canonical URLs and for searching and filtering sets of resources by various criteria. All representations are encoded as JSON objects.
You can access six resource types using the API:
Comics: individual print and digital comic issues, collections and graphic novels. For example: Amazing Fantasy #15.
Comic series: sequentially numbered (well, mostly sequentially numbered) groups comics with the same title. For example, Uncanny X-Men.
Comic stories: indivisible, reusable components of comics. For example, the cover from Amazing Fantasy #15 or the origin of Spider-Man story from that comic.
Comic events and crossovers: big, universe-altering storylines. For example, Infinity
Creators: women, men and organizations who create comics. For example, Jack Kirby.
Characters: the women, men, organizations, alien species, deities, animals, non-corporeal entities, trans-dimensional manifestations, abstract personifications, and green amorphous blobs which occupy the Marvel Universe (and various alternate universes, timelines and altered realities therein). For example, Spider-Man.
In case you are comics lover developer just give them a try:
https://developer.marvel.com/docs
Announcement for subscribers from Russia:
Это не реклама, просто дружественное размещение в личном канале.
27 октября проводится очередной ITsubbotnik в Москве (Москва, DEWORKACY, Красный Октябрь, Берсеневская наб., 6 строение 3).
https://events.epam.com/events/itsubbotnik-technological-mix?utm_source=telegram&utm_medium=cpc
Темы вроде не про маркеринг, просмотрите тут:
https://events.epam.com/events/itsubbotnik-technological-mix/agenda
Guido van Rossum steps down as Benevolent Dictator For Life of Python! So even he just burned out, eh...
https://mail.python.org/pipermail/python-committers/2018-July/005664.html
https://www.reddit.com/r/programming/comments/8yb4bo/guido_stepping_down_as_bdfl_for_python/
Nice pack for python developers:
https://www.humblebundle.com/software/python-dev-kit-bundle
classic "gaming" Humble Bundle bundles python software/books this time (from 1$). For example: half year PyCharm Pro subscription for 15$ (+ GitKraken Pro 1 Year + 2 courses + 2 books + support charity)
Hint: remember! (!) Do not rely on passwords only! In case you want secure service you must add at least 2 factors (email/phone) for authentication (password/SMS/TOTP code). Sure there is so much more...
By the way the mentioned Troy Hunt's pluralsight courses are VERY nice:
https://www.pluralsight.com/authors/troy-hunt
For example on the theme of passwords and overall security of the website: https://www.pluralsight.com/courses/hack-yourself-first
UPDATE: OnePassword took the mentioned above service's API for their proof-of-concept password check function:
https://blog.agilebits.com/2018/02/22/finding-pwned-passwords-with-1password/
"Sex toys and their nice pack of vulnerabilities"
Last year was a year of IOT vulnerabilities with Mirai botnet and its siblings. Here is 2018 and another very intimate "professional" "IOT", just look at their problems:
The following vulnerabilities, describe issues in the iOS/Android application and the corresponding backend as well as hardware related issues.
1. Customer Database Credential Disclosure
2. Exposed administrative interfaces on the internet
3. Cleartext Storage of Passwords
4. Unauthenticated Bluetooth LE Connections
5. Insufficient Authentication Mechanism
6. Insecure Direct Object Reference
7. Missing Authentication in Remote Control
8. Reflected Cross-Site Scripting
Bluetooth without pairing and allowing access your toys to others are just minor ones, just like we like them to be.
https://www.sec-consult.com/en/blog/2018/02/internet-of-dildos-a-long-way-to-a-vibrant-future-from-iot-to-iod/index.html
Short for now: udemy.com are "celebrating" 20 million students and all the courses are for just 9.99$. The good thing about their courses:
▶️ Many of the authors are from other paid services and the courses were migrated, for example I've tried their Python bootcamp and it's really easygoing and nice
▶️ There is no subscription: you just buy the course and can use it together with excersises and mobile/desktop clients as long as you like
You have 14 hours from now until the promo ends. GO 📺
A very interesting article on how one developer can "hack" open source with malicious code! It can be true/false, but there are certainly points of interest in the article. The guy did everything using NPM(!):
Lucky for me, we live in an age where people install npm packages like they’re popping pain killers.
Just inserted the code as pull request to some packages and used it for stealing valuable data from web projects.
https://hackernoon.com/im-harvesting-credit-card-numbers-and-passwords-from-your-site-here-s-how-9a8cb347c5b5
That "node community" is too heavily investing in dependencies with the culminations like this or another one, happened last year, when single developer "broke the internet" when he unpublished his modules:
http://azer.bike/journal/i-ve-just-liberated-my-modules/
The main problem there was that really SMALL module (leftpad), because of KIK messenger sue (kik https://github.com/hek/hek) has broken many dependent libraries just because they've used the dependence.
That left-pad code was JUST:
module.exports = leftpad;
function leftpad (str, len, ch) {
str = String(str);
var i = -1;
if (!ch && ch !== 0) ch = ' ';
len = len - str.length;
while (++i < len) {
str = ch + str;
}
return str;
}
https://www.theregister.co.uk/2018/01/02/intel_cpu_design_flaw/
In short:
There is security bug in Intel produced processors and the only way to fix it are changes in Linux/Windows kernels! It allows normal user programs – from database applications to JavaScript in web browsers – to discern to some extent the layout or contents of protected kernel memory areas.
Ou.... 15-30 slowdown of Intel based processors is a really huge deal! Lets see where this news will lead, but for now 2.01.2017 is a very bad day for Intel (good for AMD).
AMD processors are not subject to the types of attacks that the kernel page table isolation feature protects against. The AMD microarchitecture does not allow memory references, including speculative references, that access higher privileged data when running in a lesser privileged mode when that access would result in a page fault.
Google just launched its BETA (for Google products are mostly infinitely in beta stage) of the natural language API. For now it can work with English, Spanish or Japanese. You can use it to extract information from the text automatically about places, events, people and more to build decisions based on that models.
https://cloud.google.com/natural-language/
https://techcrunch.com/2016/07/20/google-launches-new-api-to-help-you-parse-natural-language/
Microsoft has published REST API design guidelines they use inside the company. Defenetely worth reading: http://dev.office.com/blogs/rest-api-design-guidelines
Читать полностью…
Seems like a nice resource for training in AI: https://gym.openai.com/ . There are samples with agents that can play pong and go.
OpenAI Gym is compatible with algorithms written in any framework, such as Tensorflow and Theano. The environments are written in Python, but we'll soon make them easy to use from any language. We originally built OpenAI Gym as a tool to accelerate our own RL research. We hope it will be just as useful for the broader community.
AFTER LOSING ON MOBILE, CAN MICROSOFT WIN THE NEXT BATTLE?
AI / NCL are trends right now (personally I think that they are more closer to average people, unlike IOT, as a services, of course, not as complex math models), so this longread should be interesting to read.
Microsoft argues that it has the best "brain," built on nearly two decades of advancements in machine learning and natural language processing, for delivering a future powered by artificial intelligence. It has a head start in building bots that resonate with users emotionally, thanks to an early experiment in China. And among the giants, Microsoft was first to release a true platform for text-based chat interfaces — a point of pride at a company that was mostly sidelined during the rise of smartphones.
http://www.theverge.com/2016/7/7/12111028/microsoft-bot-framework-artificial-intelligence-satya-nadella-interview
To comment or not to comment... Yes, its personal, but the advice to write small micro functions with proper naming for the single action is really good practice for proper design of the whole modules. So.. If you can write the comments for some block of code - may be its better to refactor it into a method with proper title that is reflecting the previously written comments (sure it does ;))?
https://blog.codinghorror.com/coding-without-comments/
You wanted to know if regular programming can be rocket science?.. Here you go: source code of the original Apollo 11 mission on GitHub https://github.com/chrislgarry/Apollo-11
Читать полностью…
Every byte of a TLS connection explained and reproduced.
Such a nice visual representation of how TLS 1.2 is working in The Illustrated TLS Connection! 🗝
https://tls.ulfheim.net/
Github is now using HaveIBeenPwnd too: https://blog.github.com/2018-07-31-new-improvements-and-best-practices-for-account-security-and-recoverability/
Several years ago, security researcher Troy Hunt sought to tackle the compromised passwords problem with his HaveIBeenPwned.com project. While Troy hosts a service that people and services can use to check for compromised passwords, he also generously made the approximately 517 million record dataset available for download. Using this data, GitHub created an internal version of this service so that we can validate whether a user’s password has been found in any publicly available sets of breach data. Starting today, people using compromised passwords will be prompted to select a different password during login, registration, or when updating their password.
It was a bit of a surprise for me to guess that there is not much material on using https://github.com/kennethreitz/records with MySQL database… So just a smaaaaaall instruction right here:
Records is a wrapper module over https://www.sqlalchemy.org ORM for Python. So from here all the almost settings it can have are from SQLAlchemy, just with addition of some bells-and-whistles.
The basic example on Records' GitHub is:import recordsWithout any explanation :) SO this small post is just to give you the start with this nice little(not that little actually, because it adds bunch of export abilities too (whistles from Pandas)) module. The format of basic DB connection for MySQL is for example:
db = records.Database('postgres://...')
rows = db.query('select * from active_users') # or db.query_file('sqls/active-users.sql')mysql://USER:PASSWORD@HOST/DATABASE where mysql:// is driver part, so to install all the modules for MySQL you will need to install Records and MySQL client, so the final instructions for Python3 are (using pipenv, here):pipenv install records[pandas] That’s it, go ahead and try basic examples from https://github.com/kennethreitz/records or with your own db now.
pipenv install mysqlclient
* As article on Medium:
https://medium.com/emoney-engineering/super-basic-mysql-with-records-for-python-83e39c408ba6
In case you "python" - the name Kenneth Reitz is probably familiar to you. The guy is famous for his high quality open source libraries among which there are: requests(for working with HTTP requests), httpbin(for refining HTTP request/responses), pipenv(this one is actually "steroided" package manager)… Several days ago he published a new repository "requests-html" http://html.python-requests.org . The thing is a beast for parsing HTML pages with 1 key difference(not counting the nice docs/syntax for humans) from the "Zoo" of the similar libraries: it claims full javascript support! Actually it downloads Chromium(once, when needed to the '~/.pyppeteer' dir), reloads the response there and replaces the content of the raw html with the addition of content that was rendered with javascript! The potential of this thing is that you can get dynamic parts of the responses. And that is great for the websites that need to be scrapped(cuz we need sometimes information from such) with the ability to render dynamically hidden(for example) parts.
For now the feature list is:
• Full JavaScript support!
• CSS Selectors (a.k.a jQuery-style, thanks to PyQuery).
• XPath Selectors, for the faint at heart.
• Mocked user-agent (like a real web browser).
• Automatic following of redirects.
• Connection–pooling and cookie persistence.
• The Requests experience you know and love, with magical parsing abilities.
He also has "demo" project that uses the newly published scrap lib:
Twitter Scraper
Twitter's API is annoying to work with, and has lots of limitations — luckily their frontend (JavaScript) has it's own API, which I reverse–engineered. No API rate limits. No restrictions. Extremely fast.
You can use this library to get the text of any user's Tweets trivially.
Very useful for making markov chains.
https://github.com/kennethreitz/twitter-scraper
Additional sources:
http://kennethreitz.org/
https://github.com/kennethreitz
The great Troy Hunt announces the 2 version of his service "Have I Been Pwnd"
https://haveibeenpwned.com/
The service, where you can check the services/websites which leaked your password by email AND the popularity of your passwords… To not send "real" password you can send the SHA1 hashed strings of passwords and the service will return if you were pwnd (See below about more secure way to do that via API). Also there are torrents of leaked databases. The interesting part is also that the service has API with methods like:
Searching by range
In order to protect the value of the source password being searched for, Pwned Passwords also implements a k-Anonymity model that allows a password to be searched for by partial hash. This allows the first 5 characters of a SHA-1 password hash (not case-sensitive) to be passed to the API (testable by clicking here):
GET https://api.pwnedpasswords.com/range/{first 5 hash chars}
Example: https://api.pwnedpasswords.com/range/21BD1
And it returns the list of hashes, so that you recheck user's entered password with the list. That way you can filter too popular passwords and, for example, encourage the user to "harden" the password.
Go check the tool, it’s worth the time to play with the DB a bit.
Joel Spolsky.
In case you know that name - probably you already know what am I going to suggest! But for the rest of you:
Joel Spolsky is the guy behind MS Excel in early 90-s, StackOverflow/StackExchange and Trello. Sure he has more projects, but these are most popular ones, where he was working on some leading roles (Program Manager/CEO/Founder). And he ran very interesting blog, that was even published as a book. In the blog he touches different themes that are related to software development and life in general (For example I need to hire a developer… Where to find the reasonable advice?? https://www.joelonsoftware.com/2006/10/25/the-guerrilla-guide-to-interviewing-version-30/ (like: I was already in "The Simpsons")). The blog was abandoned for several years, but from time to time he still manages to write there. You really can just start reading the blog every day from the first article, he has nice vocabulary and sense of humor. Sure it will be valuable time investment, just try, that’s not that another common blog.
https://www.joelonsoftware.com
BitBucket is struggling second day from service outages already… They've even lost some valuable user data (as another information source says)! But did anyone notice? That’s the beauty of decentralization: main server is down, but we continue our work as almost nothing changed.
https://status.bitbucket.org/
...And about recent CPU problems here is "promo" website where you can find all the information, together with research papers
Bugs in modern computers leak passwords and sensitive data.
Meltdown and Spectre exploit critical vulnerabilities in modern processors. These hardware bugs allow programs to steal data which is currently processed on the computer. While programs are typically not permitted to read data from other programs, a malicious program can exploit Meltdown and Spectre to get hold of secrets stored in the memory of other running programs. This might include your passwords stored in a password manager or browser, your personal photos, emails, instant messages and even business-critical documents.
https://meltdownattack.com/
Happy 011111100010 year to all the subscribers, wish you all the best, let the new year be even more interesting journey for you! To 2018 and beyond! 🍷
Читать полностью…
A nice addition from Bitbucket: Git LFS (Large File Storage). Now we can store even large binaries without growing the size of the actual repository: bitbucket stores the large files in the cloud and the references to them in repository, so that when you are checking out some version - it will pull only the relevant binary for that version.
Why should you care about Git LFS?
Git was optimized for source – it’s easily merged and compressed and is relatively small, so it is perfectly feasible to store all history everywhere, but this makes it inefficient and slow when trying to track large binary files. For example, If your designer stores a 100 MB image in your Git repository and modifies it nine times, your repository could bloat to almost 1 GB in size, since binary deltas often compress poorly. Every developer, build agent, and deploy script cloning that repository would then have to download the full 1 GB history of changes, which may lead to drastically longer clone times. Just imagine what would have happened if your designer made 99 changes to that file.
https://blog.bitbucket.org/2016/07/18/git-large-file-storage-now-in-bitbucket-cloud/
Here are not that obvious rules facts about threading in .NET. Saved the article, because there was a moment in my career when one of them saved me lots of time (bugs in multi-thread are sometimes hard to catch before going to production in case you don't normally understand the subject).
Have you ever spent a few hours on trying to debug a non-deterministic problem occurring in your multi-threading application? If so, then you should definitely read this article. If not it is anyway a good way of revising your current knowledge about threading challenges in C#. Being aware of some common facts about threading can help you considerably in building well-designed, error-proof multi-threading applications in the future.
http://blog.goyello.com/2014/01/21/threading-in-c-7-things-you-should-always-remember-about/
Epic article about how relational databases work in general. Easy to read and VERY valuable, just because of the fact that in depth there is too much academic math and raw documentations where you must know what are you searching for when you are reading it (sure its OK, but not for newbies). http://coding-geek.com/how-databases-work/
Читать полностью…
Not directly related to the IT, but related to the information we need to process. Rush is almost always a loose when you want to understand and learn something new. The bad news is that we are already spoiled by 'scanning' through Facebook/Twitter posts.
With 1,500 – 2,000 TV shows aired, 600,000 – 1 million books published, 1 billion active websites & approximately 200 billion tweets posted every year, we live in a world packed with information. In our pockets, a thumb-press away, we carry libraries so vast that even imagining them would be an impossibility... https://blog.todoist.com/2016/04/25/read-less-learn-more/
Some recommendations for the developers of any skillset:
Try https://www.manning.com/books/c-sharp-in-depth-third-edition
in case the "depth" is the right word for your curiosity in C#.
If you're a .NET developer, you'll use C# whether you're building an advanced enterprise application or just slamming out a quick app. In C# 5 you can do amazing things with generics, lambda expressions, dynamic typing, LINQ, iterator blocks, and other features. But first you have to learn it in depth.
C# in Depth, Third Edition has been thoroughly revised to cover the new features of C# 5, including the subtleties of writing maintainable asynchronous code. You'll see the power of C# in action, learning how to work with high-value features that you'll be glad to have in your toolkit. And you'll learn to avoid hidden pitfalls of C# programming with the help of crystal clear explanations of "behind the scenes" issues.
In short: this one is the best human readable book about the language from the #1 stackoverfloow .NET superhero Jon Skeet.
A nice article in russian about a story of indie game company, who tried to be self published in XBOX/Steam Greenlight
https://m.habrahabr.ru/post/302848
