1. Got a question or need help troubleshooting? Post to the troubleshooting forum or Search the forums!

Discussion: Marlin based web configuration tool

Discussion in 'Projects' started by Sean Carson, Jun 7, 2017.

  1. Sean Carson

    Sean Carson Member

    Joined:
    Feb 4, 2016
    Messages:
    155
    Likes Received:
    22
    I have found that some other firmwares, like aprinter and repetetier, have a web config page. You pop in your settings and it spits out a pre-configured set of files to upload.

    I haven't seen that with Marlin. Do they have a web config too?
     
    #1 Sean Carson, Jun 7, 2017
    Last edited by a moderator: Jun 8, 2017
  2. daniel871

    daniel871 Well-Known Member

    Joined:
    Apr 18, 2015
    Messages:
    1,322
    Likes Received:
    510
    Caveat Emptor, but yes: https://marlinbuilder.robotfuzz.com/

    You kinda have to know what to put everywhere for it just like with manual Arduino editing, and you still have to manually flash it with Arduino, so Waldo's work isn't invalidated. :)
     
    Sean Carson likes this.
  3. WheresWaldo

    WheresWaldo Volunteer ( ͠° ͟ʖ ͡°)
    Staff Member

    Joined:
    Feb 18, 2015
    Messages:
    5,806
    Likes Received:
    3,521
    Marlin builder is based on RC6 so it is very far behind current code.

    There is another one to, called QTMarlin but it is 6 years old. It is even further behind.

    Once you edit enough Configuration.h files it becomes easier. Besides most people don't replace firmwares every time there is a new build.
     
    #3 WheresWaldo, Jun 7, 2017
    Last edited: Jun 7, 2017
    Robert55 and Sean Carson like this.
  4. bamhm182

    bamhm182 Active Member

    Joined:
    Mar 17, 2014
    Messages:
    167
    Likes Received:
    86
    I bought a domain planning to create some php based gcode modifiers and have yet to do anything with it. I was thinking about hosting some firmwares for printers that are relevant to me. I don't think it would be that big of a leap to modify it to create custom firmwares too. If there is enough interest in this, I'll have a go at it. I could do something similar to the marlin builder and work with WheresWaldo to figure out what exactly needs to be changed. I can think of a super quick and dirty way which could work in only a handful of hours. I'd have to think longer to come up with a proper way to set it up.
     
    Sean Carson likes this.
  5. bamhm182

    bamhm182 Active Member

    Joined:
    Mar 17, 2014
    Messages:
    167
    Likes Received:
    86
    I was thinking I would just try and write something that would periodically download the firmware either automatically or manually from the github, then extract certain values to show as defaults on a webpage, then it would allow you to change them or select from various pre-configured profiles and then when you press save it allows you to download a modified firmware zip. Are you suggesting something more than that?
     
  6. Sean Carson

    Sean Carson Member

    Joined:
    Feb 4, 2016
    Messages:
    155
    Likes Received:
    22
    I was thinking you put in values and tell it what firmware you wanted, then it would make it fit. Make one profile that be be used on anything but your thing sounds more useful.

    I wonder if that could be made into a script and flashed via octoprint. Imagine a 3d printer than can just upgrade/restore itself with the octoprint webUI.
     
  7. bamhm182

    bamhm182 Active Member

    Joined:
    Mar 17, 2014
    Messages:
    167
    Likes Received:
    86
    So you'd be able to select Repetier, Marlin, or Sailfish? I think I'll stick to trying to get one firmware to work properly before trying several.

    I was playing around with something in octoprint that allows you to upload a precompiled hex files. I ended up having like 6 different firmwares I could flash for different things. If I recall, it worked pretty well, but I haven't swapped my hotend recently, so I haven't needed to use it.
     
    Sean Carson likes this.
  8. Sean Carson

    Sean Carson Member

    Joined:
    Feb 4, 2016
    Messages:
    155
    Likes Received:
    22
    To the first point, yes, but as pointed out earlier, most people don't tamper like I do so while cool, I don't know if others could appreciate it.

    On the other hand, your approach would give less-experienced users access to enhancements and new features. Since the pi is a totally separate computer from the arduino, it can reflash the thing if anything ever gets really screwed up.

    Oh! You could pass around printer profiles too. This brainstorming is yielding cool ideas.
     
  9. bamhm182

    bamhm182 Active Member

    Joined:
    Mar 17, 2014
    Messages:
    167
    Likes Received:
    86
    Well, I was hoping to have the code extract all the things WheresWaldo changed before going to bed, but man, you've been busy! I've still got around 30 more in the Configuration.h, but this is what I've got so far:

    <Changed things around. Links broke and there's no point in replacing them ATM>

    So far I'm just working on having it extract the values from the files. After that, people will be shown a nice UI to change them as they see fit, then it will let them download the changed file.
     
    #9 bamhm182, Jun 8, 2017
    Last edited: Jun 13, 2017
    Sean Carson likes this.
  10. WheresWaldo

    WheresWaldo Volunteer ( ͠° ͟ʖ ͡°)
    Staff Member

    Joined:
    Feb 18, 2015
    Messages:
    5,806
    Likes Received:
    3,521
    @bamhm182 My edits are more than just making it work. The problem is that Marlin is moving through a transition period. This is why all attempts to do exactly what you are doing have failed in the past. It isn't just filling in numbers from iteration to iteration. Right now Marlin in preparing to rebase all of their code. As a result Defines and Variable names change, sometimes within the very same version. Things get moved around in Configuration .h and you have to account for that. There are features that come and go, sometimes even before you realize they were there. Features are being deprecated. Combinations of features that may work in a version one week may not in the same set of files a week later.

    This and more is why you will be hard pressed to build a one and done configuration tool. This is why Marlin includes sample configurations to help you with supporting a particular printer.

    Then there are my additional edits. Those I include because I don't want to type them in every single variant. They won't help you and are actually unneeded to make Marlin run but make my life infinitely easier. As an example, R1 ~ R1 ~ R1 so I have multiple lines already in Configuration.h that have the proper settings for the number of steps and then uncomment the one for the version of the R1 I am dealing with. But that isn't all, if you are using mesh there are differences between R1's since I want to keep the adjustment moves on full step boundaries, So the R1 with 8 mm threaded rods move 0.025 mm per adjustment, the R1+PLUS moves 0.020 mm and if you modified it with parts like a TR8*2 lead screw (as I have on my printer) the adjustment is 0.010 mm for each click. There are also a few miscellaneous edits just because the Marlin defaults won't work if enabled with the R1. There are even some things I don't edit, for example @danzca6 uses filament change and I don't enable it, some use RGB or LED lighting and I tested it in one set but stopped enabling it after that. It's a lot to keep track of.

    Now we move on to Configuration_adv.h and there are a few more edits in that file, for some reason babystepping is not enabled/disabled in Configuration.h. You also can't babystep X and Y but they are included by default. Fortunately they do not need to change between R1 variants. Then on top of that, to keep things tidy I also edit Version.h so that if you own an R1 it doesn't say R1+ ready! when you start up your printer with an attached LCD.

    On top of that you have to make sure it all compiles without errors, else people will not use your tool, you could compile them on your host and just provide a .hex file that can be flashed to the Robo, but most users won't know what to do with that. That is the format MatterControl; and Cura and an OctoPrint add-on use, a pre-compiled binary, when you choose update firmware from those applications. If I had wanted to keep my edits to myself I would have just provided a flashable .hex file instead of all the source code sets. I want people to learn if they desire how to configure Marlin for themselves.

    One thing in your favor, I wanted to make sure that people could see my edits, so I commented every line I changed from Marlin's default settings. Somewhere, either the beginning (if the comment is solely mine) or at the end I did add --BH to each line. so a DIFF should be able to show you all the changes, What makes that still unreliable is that is only guaranteed to work on file sets downloaded on the very same day I chose to download them. This is a direct result from the source code in such a state of flux.

    More power to you and I will help where I can (limited and extremely valuable resource called time). But you will find like all the other configuration creators before you that you are chasing a tiger's tail and the constant work just to keep current will kill your will to live. Frustration when a combination of features works today but doesn't work tomorrow will make you loathe humanity.

    Now with regard to all those other firmwares that can do this but Marlin can't/won't proclaimers, nearly all of them are single contributors so one person has full control over every aspect of the code. They know what changed and why it was changed and know that a configuration tool they created has to match. Marlin contributors are following their contributions to Marlin threads and even if they know about such a tool, when there is a material change you will have to stumble through the dark to find it. Just an example was my recent spat with the contributors about deprecating MESH Bed Leveling, there are over 70 posts in that one issue, with 4 different code contributors putting their own spin on the solution needed. Finally after all was said and done, only one of the four understood the actual need, and the head contributor pipes in at the end and suggests that MESH was so small that maybe they could keep it (that actually will not happen). You will need to keep track of all those shenanigans.

    Oh, by the way, the reason most open source printers and some not so open use Marlin is because it supports much more hardware than Teacup and Repetier and Sprinter and others. It also has the most vigorous developers/code contributors. Can you get the other firmwares to work, well most likely yes. But people will soon start to complain that I want to use this new feature but T or S or R don't support it and it could be a long wait till it does or it may never support it. Not one of them uses MESH or BILINEAR bed adjustment layers. So what I would tell people that want to use limited firmware like those go back to Robo provided Marlin 1.0.0, it works and has every feature those alternatives have and there is no work necessary.

    Before you suggest that I am just a Marlin fanboy, please note that I complain about a lot of the shit that they pull and think that they most times do no regression testing any of their new code. And honestly, I would switch to something else in a heartbeat, if there were truly anything better and under active development. And one last note, My primary motivation for doing this was that I wanted newer firmware, so I set things up with parameters that work on my R1. They just happen to work for 99% of the other R1's out there too, since I was editing configurations anyway, added those few more.

    I know this is a lot to read and may sound like a rant, it is intended to inform, so I'll shorten it for those that don't want to read the whole post:

    tldr; Good luck, and I hope you can keep up with all the changes being made to Marlin on a near daily basis.
     
    #10 WheresWaldo, Jun 8, 2017
    Last edited: Jun 8, 2017
  11. bamhm182

    bamhm182 Active Member

    Joined:
    Mar 17, 2014
    Messages:
    167
    Likes Received:
    86
    @WheresWaldo, these are some very good points you bring up that I didn't take into consideration because of just jumping into this and not taking the time to look into the state of Marlin... It still seems do-able, but just with more work. I mean, I imagine the core functionality is going to be the same. I imagine that PID, bedPID, max/min temps/acceleration, etc. are going to exist in every version, even if the names change like you had said. Additionally, these things will always be set with the same format of #define variable value unless arduino goes under a massive change. I'm wondering if maybe it would be better to just have a weekly download of the tool and a file that translates variables used in each download to my variables. That would, however, mean that this is something I would have to work on every week, or however often I would decide to update them. I could probably make my life a bit easier by having another process to scan the files for the variables I'm looking for and create that file as best it can. I'm sure it'll still require some human intervention though. Hmn... I can see why the other attempts are dead all the sudden. >_<

    Maybe what I should be doing first is downloading the firmware that Robo provides and fixing it to compile with the most recent version of Arduino, then making a universal configuration tool for that. Since we know it works, we know what needs to change, and we know it won't go changing daily.

    The problem of making sure the code will compile is an easy enough one to solve. I imagine I could just install the Arduino SDK on my server and do a test build before they can download it. I would also need to put in a ton of input validation to make sure people aren't doing anything screwy.

    I did see all of your BH comments. Very much appreciated as it made my job quite a bit easier so far.
     
  12. WheresWaldo

    WheresWaldo Volunteer ( ͠° ͟ʖ ͡°)
    Staff Member

    Joined:
    Feb 18, 2015
    Messages:
    5,806
    Likes Received:
    3,521
    @bamhm182 Most of what you are asking is already done by GITHUB and it's Travis system with regard to compilation and testing. There is already a function to sanity check all the parameters. Look at that for automated checking. If your tool is searching for specific strings such as variable names then every change, and there have been a lot recently, would require a tool update. They will still be #define Yes a lot of stuff remains the same for each version. Rarely, but it does happen, a parameter value changes, where one number might be appropriate for one set of files, that same value might not work for another. Sometimes stuff just goes away. I don't even release files weekly, because it is of very little value for a couple of reasons. Some stuff doesn't matter to R1 users, some stuff works differently or breaks from one week to the next. An example of broken stuff, on May 17th MESH worked as it always worked in the past, on June 4th the exact same parameters would not compile. The developers claimed that there was no change to the code, it simply didn't work. A few days later they changed their Sanity check to prevent the same combination of features from being selected together. That is what you have to keep up with and no algorithm you write will account for such human behavior.

    The old Robo version of Marlin is basically just 1.0.0 with a bunch of specific configuration items, no code or functionality was changed, the few pieces of Marlin code they did try to adjust @danzca6 and I figured out were broken and didn't execute, The issue with new compilers and old code was a conflict with a specific library (the SD file handling) and was a result of changes in that library, a simple edit and replace of a few lines of code makes the old code compliant with newer Arduino compilers, why Robo has not done this themselves I cannot answer, I belief is that they should have done this themselves. Since the old Robo code is already supplied in a form that each printer needs, I am not sure where your work in creating a tool for it will add any value and I suspect your time is very valuable. Besides that, the basis for that code is a now very outdated version of Marlin. The real value would be in current versions of Marlin, but I think I outlined my concerns about how much work that will be.
     
  13. bamhm182

    bamhm182 Active Member

    Joined:
    Mar 17, 2014
    Messages:
    167
    Likes Received:
    86
    The more I'm thinking about this, the more I'm beginning to agree that my idea is a lost cause. Probably should have done my research before trying to jump into this. Is there not some stable branch of Marlin? Normally you'll have a stable version that is just slightly out of date, then all of the code-breaking changes happen in a daily branch or something to that likeness. If there were a stable branch that only got updated every once in a while, I think my project would still be worthwhile to do for that.

    As far as the original code not compiling, I completely agree this is something the Robo team should have addressed forever ago. I forget exactly when, but it must have been well over a year ago that I also fixed their firmware so it would compile on newer versions of Arduino. Took no more than an hour to do. I posted my non-custom, but fixed version of their official firmware on the comments section of their firmware page, but then it got deleted and I didn't feel like asking about it. Then I went on and made a version of the firmware that has a bit of variable setting at the beginning of the Configuration.h file that replicate throughout the rest of it. That way, I can change one variable to define what setup my printer is in (R1/R1++ w/ Cyclops/Volcano/Hex/etc) and put in the option of modifying the relevant variables manually. I was just thinking it would be nice to have a tool like that without having to actually go into the firmware.
     
  14. Sean Carson

    Sean Carson Member

    Joined:
    Feb 4, 2016
    Messages:
    155
    Likes Received:
    22
    I disagree. While there may b lots of changes coming, you could certainly set the stage for the final version. That way, you will be ready for whatever marlin ends up as. I really think you have a great idea. If I could, I would be glad to help, but HTML and such are beyond my skill set. I can read it, but not write it. I don't understand python either. I'm just stuck with arduino and what little C++ and Java I remember from high school.

    I'm sure there would be others here who would join you and help.
     
  15. WheresWaldo

    WheresWaldo Volunteer ( ͠° ͟ʖ ͡°)
    Staff Member

    Joined:
    Feb 18, 2015
    Messages:
    5,806
    Likes Received:
    3,521
    @bamhm182 Marlin doesn't believe in traditional development cycles, There Release Candidates are not frozen code bases, where you would just look for bugs and try to fix them all before going GOLD. Rather they seem to be more like, "maybe it's time to have a release, lets call today's build a RC". They continue to test new code and include new features even in the RC cycle. Then when they feel compelled the will say, "it's stable enough let's call today's build a release!" There is not enough time to include any testing of the previous day's code changes but they release it anyway.

    So the short answer, there is no stable branch of Marlin.

    @Sean Carson It's just too much work, that is why you wont get what you want, of course if it's that important to you and you are offering your time and programming skills to get it done, then that is different.

    So in short, there is no final version of Marlin, that gets an official release.
     
    bamhm182 and Sean Carson like this.
  16. Sean Carson

    Sean Carson Member

    Joined:
    Feb 4, 2016
    Messages:
    155
    Likes Received:
    22
    Hence, my aversion to marlin. Really, I think this is still possible. Maybe create profiles unique to each version of marlin, maintained by community. Example: app setting A. marlin equivalent is Q. Version profile says A is mapped to Q.
     
  17. bamhm182

    bamhm182 Active Member

    Joined:
    Mar 17, 2014
    Messages:
    167
    Likes Received:
    86
    It sounds like they have a promising future in the military. "Here's this technology that we say can do all of these amazing things in the future! Some of them are implemented now!" "How many of them are implemented now?" "Well it's hard to say, because it is changing daily!" "Changing daily? How often do you test?" "Never!" "Great! I'll take 12!"
     
    Sean Carson likes this.
  18. WheresWaldo

    WheresWaldo Volunteer ( ͠° ͟ʖ ͡°)
    Staff Member

    Joined:
    Feb 18, 2015
    Messages:
    5,806
    Likes Received:
    3,521
    Again, there are no real versions, are you suggesting a community takes on this challenge on a daily basis. I am not joking about this, there are substantial changes to code locations, features, coding style that happens almost every single day! There have been times when I will download code every day for a week waiting on something to work and trying different things that are broken.
     
  19. WheresWaldo

    WheresWaldo Volunteer ( ͠° ͟ʖ ͡°)
    Staff Member

    Joined:
    Feb 18, 2015
    Messages:
    5,806
    Likes Received:
    3,521
    Okay, I edited post one and cleaned up all the snarky comments, including my own.I think this is an important discussion and should continue.
     
    Geof and mark tomlinson like this.
  20. bamhm182

    bamhm182 Active Member

    Joined:
    Mar 17, 2014
    Messages:
    167
    Likes Received:
    86
    After mulling it around for a few days, I think I've figured out how I'd like to go about this project. The best way I can think of is to do something like I did with my All-In-One firmware based on the original Robo firmware.
    //==================================================================================
    //======================== Printer Specifications (bamhm182) =======================
    //==================================================================================

    // 0 = No Preset
    // 1 = R1 | E3Dv6 | Greg's Wade
    // 2 = R1++ | Hex | Bondtech
    // 3 = R1++ | E3Dv6 | Bondtech
    // 4 = R1++ | Cyclops | Bondtech
    // 5 = R1++ | Volcano | Bondtech
    #define roboPreset 4

    #if roboPreset == 0 // No Preset
    #define preheatHotend_PLA 205
    #define preheatBed_PLA 60
    #define preheatFan_PLA 255
    #define preheatHotend_ABS 225
    #define preheatBed_ABS 75
    #define preheatFan_ABS 0

    // 1 = R1
    // 2 = R1++
    #define roboVersion 1

    // 1 = Greg's Wade
    // 2 = Bondtech
    #define roboExtruder 1

    // 1 = Hex
    // 2 = E3Dv6
    // 3 = Cyclops
    // 4 = E3Dv6 Lite w/ Volcano
    #define roboHotend 2

    // 1 = Smart LCD Controller XXL (R1)
    // 2 = Full Graphics Smart Controller (R1++)
    #define roboLCD 1
    // 1 = Robo Stock
    // 2 = E3Dv6 Stock
    // 3 = Stud
    #define roboThermistor 3
    #elif roboPreset == 1 // R1 | E3Dv6 | Greg's Wade
    #define preheatHotend_PLA 205
    #define preheatBed_PLA 60
    #define preheatFan_PLA 255
    #define preheatHotend_ABS 225
    #define preheatBed_ABS 75
    #define preheatFan_ABS 0
    #define roboVersion 1 // R1
    #define roboExtruder 1 // Greg's Wade
    #define roboHotend 2 // E3D v6
    #define roboLCD 1 // Smart LCD Controller XXL
    #define roboThermistor 3 // Stud
    #elif roboPreset == 2 // R1++ | Hex | Bondtech
    #define preheatHotend_PLA 205
    #define preheatBed_PLA 60
    #define preheatFan_PLA 255
    #define preheatHotend_ABS 225
    #define preheatBed_ABS 75
    #define preheatFan_ABS 0
    #define roboVersion 2 // R1++
    #define roboExtruder 2 // Bondtech
    #define roboHotend 1 // Hex
    #define roboLCD 2 // Full Graphics Smart Controller
    #define roboThermistor 1 // Stock
    #elif roboPreset == 3 // R1++ | E3Dv6 | Bondtech
    #define preheatHotend_PLA 205
    #define preheatBed_PLA 60
    #define preheatFan_PLA 255
    #define preheatHotend_ABS 225
    #define preheatBed_ABS 75
    #define preheatFan_ABS 0
    #define roboVersion 2 // R1++
    #define roboExtruder 2 // Bondtech
    #define roboHotend 2 // E3D v6
    #define roboLCD 2 // Full Graphics Smart Controller
    #define roboThermistor 3 // Stud
    #elif roboPreset == 4 // R1++ | Cyclops | Bondtech
    #define preheatHotend_PLA 205
    #define preheatBed_PLA 60
    #define preheatFan_PLA 0
    #define preheatHotend_ABS 225
    #define preheatBed_ABS 75
    #define preheatFan_ABS 0
    #define roboVersion 2 // R1++
    #define roboExtruder 2 // Bondtech
    #define roboHotend 3 // Cyclops
    #define roboLCD 2 // Full Graphics Smart Controller
    #define roboThermistor 2 // E3D Stock
    #elif roboPreset == 5 // R1++ | Volcano | Bondtech
    #define preheatHotend_PLA 225
    #define preheatBed_PLA 60
    #define preheatFan_PLA 255
    #define preheatHotend_ABS 225
    #define preheatBed_ABS 75
    #define preheatFan_ABS 0
    #define roboVersion 2 // R1++
    #define roboExtruder 2 // Bondtech
    #define roboHotend 4 // E3D v6 Lite w/ Volcano
    #define roboLCD 2 // Full Graphics Smart Controller
    #define roboThermistor 3 // Stud
    #endif

    I had that near the very beginning of the Configuration.h file, then everywhere that had the variables I had set up would be replaced with my variable names. For example, #define PLA_PREHEAT_HOTEND_TEMP 210 becomes #define PLA_PREHEAT_HOTEND_TEMP preheatHotend_PLA. With a setup that works in a similar way, whenever I feel like updating the tool, I would just go online, download the version of Marlin I want to update it with, then go through the Configuration.h file and replace their numbers with my variables names. Once I do that, I add the version I just downloaded to a dropdown list on the configuration page with a name like "20170613" because they don't seem to really use numbered versioning and it can be selected. I'm thinking that if I do it this way, it should be super simple to make files like 20170613_default.cfg, 20170613_Waldo.cfg, 20170613_Robo_Cyclops.cfg, etc. that would just be a short text file with variable names and their default values and have another dropdown that would allow you to select pre-defined variable sets that you could change to your liking before compiling the firmware.

    I think very first thing's first, I'm going to work on creating one that works with the firmware provided by Robo 3D since I know it is stable enough and have had the most experience with that one, then I'll probably move onto the versions that @WheresWaldo has provided. From then, I'll decide if it's worth trying to keep up with the Marlin team from time to time.
     

Share This Page