mattata's Channel

04 Dec 2017 07:57

That's 100,000 more active users than I predicted 😆

mattata's Channel

04 Dec 2017 07:42

It's like a huge jigsaw. There are people in here with "neighbouring" IDs. There's also some dude who has an insane amount of 9s in his ID.

mattata's Channel

04 Dec 2017 07:40

It's still going too, I haven't finished sorting users out yet and, well...

mattata's Channel

04 Dec 2017 07:31

On a positive, this still means mattata has over 200 thousand active users! That's like, a really big number dude

mattata's Channel

04 Dec 2017 07:29

I'm only saving user information for 2 weeks now. Any users who don't interact with mattata for 2 weeks or longer will be forgotten (however, settings and toggled plugins will remain, this is just information such as their name, username and ID - information which is cached simply to prevent the need to repeatedly call the getChat method)

mattata's Channel

04 Dec 2017 03:14

User data is being migrated to a JSON structure, in such a way that means all user info for the user with the ID 1234 will be in a file located data/users/1234/info.json. This is really simple for noobs to understand, and it's not kept in the memory, each user is constantly being updated and this info is read when necessary. It's a lot more practical because every time mattata meets a new user, redis uses a slightly bit more memory, and I can't forever upgrade the server. Assuming I don't do anything retarded in the process, no data loss (like your settings for plugins and administration) will occur. In fact, you probably won't even notice the changes have happened.

It's ~2AM here, so I'm now going to run a script that will basically migrate all user/chat data (at the moment this is just things like name, username, ID, etc.) to this new format. This is hopefully just a temporary solution to the memory issue I seem to be facing a lot, and a permanent move to PostgreSQL should happen some time in 2018.

If you URGENTLY require mattata's functionality during the next hour, you have about 5 minutes before she goes offline, so please use a temporary alternative such as Group Butler (if you're using her for administrative purporses) or otouto (if you're just using the nifty things) [do note that there are still many features mattata boasts which, to my knowledge, no other bots other than up-to-date forks of mattata are capable of doing, but none of these are really dependant features]

mattata's Channel

27 Nov 2017 23:05

Many people have been saying they are unable to donate because of payment restrictions in their country.

I will now happily accept any donations in Bitcoin or Ether!

Bitcoin: 12ipinb7HKXYGUSk8eNeewDuBx1w2vTzFm
Ether: 0xD4e1917Eca52E50A9A30E8400E26104114d0f929

Thanks again!

mattata's Channel

27 Nov 2017 12:33

Since this date, the number of users in the database has reached over 870,000, growing quicker every day. Number of chats sits at over 58,000!

mattata's Channel

23 Nov 2017 23:01

Oh also I released telegram-bot-lua v1.8 to add support for latest bot API additions!

mattata's Channel

12 Nov 2017 21:57

I've already pushed changes which move more misc functions from mattata.lua to libs/utils.lua, and fixes a bug which broke the "delete commands" setting in /administration.

mattata's Channel

11 Nov 2017 02:16

I'm now finishing off mattata's Christmas makeover, I hope you like it!

mattata's Channel

11 Nov 2017 01:54

Redis background saving schema relies on the copy-on-write semantic of fork in modern operating systems: Redis forks (creates a child process) that is an exact copy of the parent. The child process dumps the DB on disk and finally exits. In theory the child should use as much memory as the parent being a copy, but actually thanks to the copy-on-write semantic implemented by most modern operating systems the parent and child process will share the common memory pages. A page will be duplicated only when it changes in the child or in the parent. Since in theory all the pages may change while the child process is saving, Linux can't tell in advance how much memory the child will take, so if the overcommit_memory setting is set to zero fork will fail unless there is as much free RAM as required to really duplicate all the parent memory pages, with the result that if you have a Redis dataset of 3 GB and just 2 GB of free memory it will fail.

Setting overcommit_memory to 1 says Linux to relax and perform the fork in a more optimistic allocation fashion, and this is indeed what you want for Redis.
— http://redis.io/topics/faq

mattata's Channel

08 Nov 2017 23:46

A future update will see mattata use PostgreSQL for the main database, and redis for lightweight caching. I'm currently sorting out an up-to-date Lua SQL driver (the LuaSQL project has a lot of compiling issues, it seems to be very outdated, so I might need to write my own in C)

mattata's Channel

08 Nov 2017 22:21

It seems to all work right now, so I'll work on something, and leave it be in the meantime.

mattata's Channel

08 Nov 2017 20:03

I'm looking into seeing how well mattata can run without a database!

mattata's Channel

04 Dec 2017 07:57

Jesus Christ it just passed 300 thousand

mattata's Channel

04 Dec 2017 07:41

As you can see, that's a lot of directories for a lot of users 😆

mattata's Channel

04 Dec 2017 07:32

(Active meaning they've made some kind of contact with mattata in the past 2 weeks)

mattata's Channel

04 Dec 2017 07:30

This means information for about 75% of users that mattata knows, is no longer available, since these are people who mattata has seen in the past year but maybe aren't in groups with mattata anymore, so I don't really want to consider them "users".

mattata's Channel

04 Dec 2017 03:15

TL;DR database structure update, you won't lose any data so it's nothing serious for you, but in 5 minutes mattata will have some downtime (this could be anywhere up to a few hours, if my estimations are correct)

mattata's Channel

28 Nov 2017 00:12

A new wrapper for the Telegram bot API: https://github.com/jTelegram/jTelegramBotAPI

This wrapper fully embraces Java 8, and all the glory it has to offer.

They have a support group right here on Telegram! @JTelegramSupport

mattata's Channel

27 Nov 2017 12:34

In November alone, the database has grown by over 70 thousand users! This is amazing!

mattata's Channel

24 Nov 2017 17:12

News from an upcoming project, coDesign, which I am part of. Please clap and comment, thanks.

https://artplusmarketing.com/building-a-unified-aggregator-curator-codesign-a95a6cc19cd0

mattata's Channel

23 Nov 2017 22:58

I have a new PayPal! If you ever want to send me £££, make sure you use PayPal.me/matthesketh

mattata's Channel

12 Nov 2017 21:55

I'll commit practically every minor change I make to the dev branch from now on, as the changes occur

mattata's Channel

11 Nov 2017 01:57

For those of you interested in applying this little "memory patch" which solves the issue of MISCONF making Redis stop writes on BGSAVE, feel free to just use echo "vm.overcommit_memory = 1" >> /etc/sysctl.conf && sysctl -p /etc/sysctl.conf (you'll need to use su first, unless you've bumfucked your permissions)

mattata's Channel

11 Nov 2017 01:54

I've found the ideal solution until an SQL re-write is completed!

mattata's Channel

08 Nov 2017 22:22

I mean, the only downtime was because of some surprise DigitalOcean maintenance shit on my VPS.

mattata's Channel

08 Nov 2017 22:21

The answer was not very well! Who knew that you need a database to store things! Believe me, my evening has been like trying to get money out of a vault without having a vault.

mattata's Channel

03 Nov 2017 22:23

It all works now, I'll look into moving to something like SQL soon.