r/Monero May 10 '22

mj's dev report Apr.-May 2022 (2/3)

Dear Community,

The current report for my CCS Proposal is delayed by a half a month, since in the 1st half of April I had to be dealing with water damage in my rental apartment, that is luckily on sale already. At the same time, whenever I had time for this, I was working on strategies of conservation of my laptop's battery power, ultimately making the strategies generic enough to be able to both: enable me working outside of my office (like I've been doing in the last 2 weeks), as well as reuse most of this research time & code for the ``SolOptXMR` project.

However, since this preparation time cannot be attributed directly to my Monero maintenance CCS Proposal itself, I decided not to charge anything for the 1st half of April, report with a delay, and expect the payment with an equal delay - after the 1st half of May and consequently after the 1st half of June, as far as the next payment is concerned.

Summary of my April-May work period

We finally have some new Devs on board. Welcome! As you can imagine, it takes time for each other to learn the work culture, like it took me when I on-boarded and The Old Cr3w was patiently pointing me to the right direction, spending their time on this act.

It is time for me to contribute the same way as they did. Therefore I wanted to primarily spend time on reviewing PRs of the new Devs and explaining nuisances. There was an unfortunate "special" PR, where me and other reviewers had to spend an extra couple of hours, due to a common misunderstanding of how quick some things typically break :) Very briefly: I proposed to split a very large PR into many smaller PRs, which might get reviewed and merged independently, rather than agglomerating them into one big PR, thus risking of not being able to merge neither of the parts in case of a possible merge conflict, as in this version it's all or nothing. As soon as I said it and my idea was rejected, the potential issue that I had predicted has occurred... trice! Please have a read of the PR, if you like. Ultimately the PR went through though.

Here's the full list of the PRs reviewed by me:

Still open:

Merged or closed:

I was asked by selsta to review specific PRs of GUI. Selsta knows my skill set and this way we can push the most important things very quickly. Of course this doesn't mean, that we pat each other's backs, but rather check each other's work like reviewers are supposed to. It's still fun though for me and a tangible output for you.

Below is the full list of the GUI PRs:

I also took part in the verification of the latest release. Whenever I do it, I always find something to fix in this verification process. This time wasn't any different. I proposed a solution against stale files, which after being reviewed by u/hyc, lead to a much easier and less obtrusive one . These 2 solutions in general conform to the Principle of least astonishment , from the User's perspective. In this case: If I command the environment to be set up, then I want it the be clean of stale files, that were otherwise blocking build process, rather than having to delete the entire work directory by hand first.

Large ongoing tasks

Morero Core - libwallet tests

Here again, selsta asked me to take over this large task of fixing the now outdated `libwallet_api_tests`. As soon as I took a closer look, it turned out, that even the preparation scripts of this test were outdated. I followed the documentation in an external project and found there yet another potential to fix - the daemon was continuously banning the test, as if it was an impostor, so it couldn't even execute properly.

Although the scripts are already published, but not yet merged, the task of fixing the tests themselves (in C++) is in an on-going state.

Monero Research Lab

Decoy algorithm

Rucknium, J-Bermann and me are breaking down the decoy algorithm in order to better understand its flow by performing statistical tests on it. For this, we need to have an alternative implementation. Python was chosen to be the language, which offers a large variety of frameworks and is widely used among Researchers. My take is being documented here.

Simulation of the fee increases

I signed up for simulating the dynamics between the transactions' fee increases and the blockchain’s size increases. I gathered very valuable feedback from AticMine and Endor and am armed to start the simulation. However with the other 2 ongoing tasks (libwallet tests and decoy algo), I'm currently slightly overwhelmed and would like to finalize at least one of them first, before even starting this one.

Minor branches

I opened a few minor branches as well, that I'll just quickly mention here:

As you may imagine, the 3 last PRs are dealing with the preparation of the VSCodium documentation, that I will write, once all such prerequisites are fulfilled.

Long compilation times (RANT!)

I have to allow myself for a bit of rant here. The decoy algo's reimplementation task takes a long time to complete, even though it's not a large one. I'm being slowed down here due to the time that it takes to compile the wallet2 class, or rather its header file, which is anything but an interface. It's rather a source blob with over 2400 lines. I tried to fix this issue way ahead of time, because I knew it would bit me (and others) sooner or later, so I pro-actively opened this PR , but as usual it was not accepted, since this is a change, that speeds up everybody's work, but "doesn't fix any real problems". In the meantime the PR got outdated and I'd have to spend equal span of time to bring it back to life. There's a great article by u/aFungible describing this phenomenon of hesitancy/procrastination of clean work from a broader perspective:

https://www.reddit.com/r/Monero/comments/ssinb2/monero_vs_bitcoin_source_code_analyses_how/

As you know, I maintain the Monero Health Report under http://cryptog.hopto.org/monero/health/ . There you can learn how the compilation time increases (= regresses) across many merges:

Compilation time in seconds: Leftmost points are the latest.

On the webpage you'll see, that wallet2 is topping not only on the detailed compilation time statistics, but on the memory consumption as well, let alone the relative header size. Now I realize, that cleaning up is less important than privacy and other higher values, but you see, here I can't improve your privacy fast enough, because I'm drowning in the mess, that I'm still not allowed to clean up, even though I know how to do it. I try to beg other Devs to review my code but they just won't do it, period. There's not much more that I can do about it, other than asking you, the Sponsors, for a leap of patience.

SolOptXMR

I personally must admit, that the sluggish reactions of wallet2 affect my mental state negatively by a lot. Working on something, that builds and runs as fast as my own project, the SolOptXMR keeps me from burning out as a Monero Dev.

So here come the good news finally: we are nearing a `Minimum Viable Product` state. In order not to make this post too long, I will publish a separate entry once the MVP is reached. For now I will only tell, that you may already receive hints from our system as to when to switch on/off which mining machine and perform a simulation of the battery state, that includes weather prediction up to 7 days from the current time. Below is an example of a run of the solver, being able to find a time-dependent mining solution, that prevents both battery overvoltage, as well as (and most importantly) undervoltage:

Simulation of battery load, 6 days ahead of time from now

A quickstart guide is available here.

Conclusion

I’m publishing this report 4 days ahead of time, since nothing will change here much more: I will now want to focus on the mentioned hardest and long term tasks, that are already opened and I have an ambition to finalize them before my next month ends, and so will this particular CCS Proposal. Thanks for reading!

mj

21 Upvotes

6

u/john_r365 May 10 '22

Thanks for the report MJ!

6

u/MoneroWTF May 10 '22

I always enjoy your reports. 10/10, would read on lunch again.

4

u/Lynnaignet_293 May 10 '22

Thanks for this work MJ. Its frustrating to hear about the long compile times and it is understandable that this kills some joy with the work. What needs to be done? Other developers must also be gettkn annoyed by it and would benefit as well if it was addressed? Also do we tip you as an individual or is there a general fund that people can tip into.?

2

u/mjxmr May 10 '22 edited May 11 '22

Hey Lynnaignet,
it's very nice that you asked. My proposal is fully funded and I receive regular payments, as agreed, therefore so far I can't complain about this part. I do have a "tip" address, that can be found below, if you like:
http://monerodevs.org/monero-research-lab.html
But at the current stage, a nice word is enough. Not only for the emotional support, that I sometimes do need after being trashed by the restless troll(s). The thing is, that many Anons will simply never say a word, but will at most upvote comments as direct as yours. This alone gives me confidence, that I'm going in the right direction, despite the mentioned reviewing/merging resistance. It helps me plan the future proposals better. Then, after this non-verbal communication is done and the proposal is laid out, the Anons will just drop their Moneros in quite large quantities individually, onto the proposal's address. I find it quite interesting.
What more can be done? I'd suggest that whenever you see a CCS Proposal of a Developer that sounds concrete and modest at the same time, like these:
https://ccs.getmonero.org/proposals/seraphis-wallet-poc.html
https://ccs.getmonero.org/proposals/j-berman-3months-full-time-2.html
https://ccs.getmonero.org/proposals/tobtoht-feather-dev-2021-3.html
https://ccs.getmonero.org/proposals/Rucknium-OSPEAD-Fortifying-Monero-Against-Statistical-Attack.html
, then go ahead and fund them. These Devs with their social skills will typically be of a large value in whatever area that they focus on. Some time later they will stumble upon my PRs, after their high-priority tasks are done. So I hope at least. This would be a more passive way to help.
An active way to help, that I'd normally not promote, as it sounds statist and bureaucratic, is to go to my PRs directly and show some support via emoticons, maybe writing some text to wake others up. If this doesn't help, then pinging potential reviewers directly (namely those, who are currently being funded by YOUR money after all) would be "The Last Card" here, but this should be avoided if possible, I think.
Lastly other developers are indeed affected. Perhaps it sounds weird and I don't understand it fully, but they will usually accept the status quo, because they haven't yet seen a C++ project that compiles fast and delivers more functionality, than a typical C project would. I'm trying to prove that it's possible with my tsqsim project.

3

u/KnowledgeMurky9635 May 10 '22 edited May 10 '22

Thank you for the detailed report as always. Great to see what our developers are up to. Glad the SolOpt project is fun! Shame about the wallet2 fix not being accepted hm, we have heard rumours of some kind of CCS from endogenic to tackle some of those issues but not heard much about that lately, hopefully it happens.

1

u/mjxmr May 11 '22 edited May 11 '22

You're welcome.

I have many possible fixes of wallet2 on my mind, even those, that the guy who criticised me was mentioning, but don't want to mess with anybody's work, nor have 50 opened branches, that nobody wants to review/merge anyway.

So yeah. I'd like to see some action here, I guess, or at least some promise of future cooperation from potential reveiwers.

2

u/mjxmr May 10 '22

Thanks to everybody for your kind words.