Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

migrate CAM parameters #3280

Closed
wants to merge 12 commits into from
Closed

migrate CAM parameters #3280

wants to merge 12 commits into from

Conversation

caco3
Copy link
Collaborator

@caco3 caco3 commented Sep 30, 2024

Added migration of CAM parameters:

[0d00h00m02s] 2024-10-01T21:53:28 <ERR> [HELPER] Migrated Configfile line 'Brightness = 0' to 'CamBrightness = 0'
[0d00h00m02s] 2024-10-01T21:53:28 <ERR> [HELPER] Migrated Configfile line 'Contrast = 0' to 'CamContrast = 0'
[0d00h00m02s] 2024-10-01T21:53:28 <ERR> [HELPER] Migrated Configfile line 'Saturation = 0' to 'CamSaturation = 0'
[0d00h00m02s] 2024-10-01T21:53:28 <ERR> [HELPER] Migrated Configfile line 'Sharpness = 0' to 'CamSharpness = 0'
[0d00h00m02s] 2024-10-01T21:53:28 <ERR> [HELPER] Migrated Configfile line 'ImageQuality = 12' to 'CamQuality = 12'
[0d00h00m02s] 2024-10-01T21:53:28 <ERR> [HELPER] Migrated Configfile line 'ImageSize = VGA' to ';UNUSED_PARAMETER = VGA'
[0d00h00m02s] 2024-10-01T21:53:28 <ERR> [HELPER] Migrated Configfile line 'Zoom = false' to 'CamZoom = false'
[0d00h00m02s] 2024-10-01T21:53:28 <ERR> [HELPER] Migrated Configfile line 'ZoomMode = 0' to 'CamZoomSize = 0'
[0d00h00m02s] 2024-10-01T21:53:28 <ERR> [HELPER] Migrated Configfile line 'ZoomOffsetX = 0' to 'CamZoomOffsetX = 0'
[0d00h00m02s] 2024-10-01T21:53:28 <ERR> [HELPER] Migrated Configfile line 'ZoomOffsetY = 0' to 'CamZoomOffsetY = 0'
[0d00h00m02s] 2024-10-01T21:53:28 <ERR> [HELPER] Migrated Configfile line 'Grayscale = false' to ';UNUSED_PARAMETER = false'
[0d00h00m02s] 2024-10-01T21:53:28 <ERR> [HELPER] Migrated Configfile line 'Negative = false' to ';UNUSED_PARAMETER = false'
[0d00h00m02s] 2024-10-01T21:53:28 <ERR> [HELPER] Migrated Configfile line 'Aec2 = false' to 'CamAec2 = false'
[0d00h00m02s] 2024-10-01T21:53:28 <ERR> [HELPER] Migrated Configfile line 'AutoExposureLevel = 0' to 'CamAeLevel = 0'
[0d00h00m02s] 2024-10-01T21:53:28 <ERR> [HELPER] Migrated Configfile line 'FixedExposure = false' to 'CamAec = false'
[0d00h00m02s] 2024-10-01T21:53:28 <ERR> [HELPER] Migrated Configfile line 'InitialMirror = false' to ';UNUSED_PARAMETER = false'
[0d00h00m02s] 2024-10-01T21:53:28 <ERR> [HELPER] Migrated Configfile line 'FlipImageSize = false' to ';UNUSED_PARAMETER = false'
[0d00h00m02s] 2024-10-01T21:53:28 <ERR> [HELPER] Migrated Configfile line 'main.AnalogDigitalTransitionStart = 9.2' to 'main.AnalogToDigitTransitionStart = 9.2'
[0d00h00m02s] 2024-10-01T21:53:28 <ERR> [HELPER] Migrated Configfile line ';main.Fieldname = undefined' to ';main.Field = undefined'
[0d00h00m02s] 2024-10-01T21:53:28 <INF> [MAIN] Config file migrated. Saved backup to /sdcard/config/config.bak

See #3277

@caco3 caco3 requested a review from SybexX September 30, 2024 21:55
@caco3 caco3 linked an issue Sep 30, 2024 that may be closed by this pull request
@caco3 caco3 marked this pull request as draft September 30, 2024 22:09
@SybexX
Copy link
Collaborator

SybexX commented Oct 1, 2024

@caco3 I implemented it this way for myself:

void migrateConfiguration(void)
{
    bool migrated = false;

    if (!FileExists(CONFIG_FILE))
    {
        LogFile.WriteToFile(ESP_LOG_ERROR, TAG, "Config file seems to be missing!");
        return;
    }

    std::string section = "";
    std::ifstream ifs(CONFIG_FILE);
    std::string content((std::istreambuf_iterator<char>(ifs)), (std::istreambuf_iterator<char>()));

    /* Split config file it array of lines */
    std::vector<std::string> configLines = splitString(content);

    /* Process each line */
    for (int i = 0; i < configLines.size(); i++)
    {
        // ESP_LOGI(TAG, "Line %d: %s", i, configLines[i].c_str());

        if (configLines[i].find("[") != std::string::npos)
        {
            // Start of new section
            section = configLines[i];
            replaceString(section, ";", "", false); // Remove possible semicolon (just for the string comparison)
            // ESP_LOGI(TAG, "New section: %s", section.c_str());
        }

        /* Migrate parameters as needed
         * For the boolean parameters, we make them enabled all the time now:
         *  1. If they where disabled, set them to their default value
         *  2. Enable them
         * Notes:
         * The migration has some simplifications:
         *  - Case Sensitiveness must be like in the initial config.ini
         *  - No Whitespace after a semicollon
         *  - Only one whitespace before/after the equal sign
         */
        if (section == "[MakeImage]")
        {
            migrated = migrated | replaceString(configLines[i], "[MakeImage]", "[TakeImage]"); // Rename the section itself
        }

        if (section == "[MakeImage]" || section == "[TakeImage]")
        {
            if ((isInString(configLines[i], "Brightness")) && (!isInString(configLines[i], "CamBrightness")))
            {
                migrated = migrated | replaceString(configLines[i], "Brightness", "CamBrightness");
            }
            else if ((isInString(configLines[i], "Contrast")) && (!isInString(configLines[i], "CamContrast")))
            {
                migrated = migrated | replaceString(configLines[i], "Contrast", "CamContrast");
            }
            else if ((isInString(configLines[i], "Saturation")) && (!isInString(configLines[i], "CamSaturation")))
            {
                migrated = migrated | replaceString(configLines[i], "Saturation", "CamSaturation");
            }
            else if ((isInString(configLines[i], "Sharpness")) && (!isInString(configLines[i], "CamSharpness")) && (!isInString(configLines[i], "CamAutoSharpness")))
            {
                migrated = migrated | replaceString(configLines[i], "Sharpness", "CamSharpness");
            }
            else if ((isInString(configLines[i], "Aec2")) && (!isInString(configLines[i], "CamAec")) && (!isInString(configLines[i], "CamAec2")))
            {
                migrated = migrated | replaceString(configLines[i], "Aec2", "CamAec2");
            }
            else if ((isInString(configLines[i], "Zoom")) && (!isInString(configLines[i], "CamZoom")))
            {
                migrated = migrated | replaceString(configLines[i], "Zoom", "CamZoom");
            }
            else
            {
                migrated = migrated | replaceString(configLines[i], "LogImageLocation", "RawImagesLocation");
                migrated = migrated | replaceString(configLines[i], "LogfileRetentionInDays", "RawImagesRetention");

                migrated = migrated | replaceString(configLines[i], ";Demo = true", ";Demo = false"); // Set it to its default value
                migrated = migrated | replaceString(configLines[i], ";Demo", "Demo");                 // Enable it

                migrated = migrated | replaceString(configLines[i], "ImageQuality", "CamQuality");
                migrated = migrated | replaceString(configLines[i], "AutoExposureLevel", "CamAeLevel");
                migrated = migrated | replaceString(configLines[i], "FixedExposure", "CamAec");

                // migrated = migrated | replaceString(configLines[i], "Zoom", "CamZoom");
                migrated = migrated | replaceString(configLines[i], "ZoomMode", "CamZoomSize");
                migrated = migrated | replaceString(configLines[i], "CamZoomMode", "CamZoomSize");
                // migrated = migrated | replaceString(configLines[i], "ZoomOffsetX", "CamZoomOffsetX");
                // migrated = migrated | replaceString(configLines[i], "ZoomOffsetY", "CamZoomOffsetY");

                migrated = migrated | replaceString(configLines[i], "ImageSize", ";UNUSED_PARAMETER"); // This parameter is no longer used
                migrated = migrated | replaceString(configLines[i], "Grayscale", ";UNUSED_PARAMETER"); // This parameter is no longer used
                migrated = migrated | replaceString(configLines[i], "Negative", ";UNUSED_PARAMETER");  // This parameter is no longer used
            }
        }
        else if (section == "[Alignment]")
        {
            migrated = migrated | replaceString(configLines[i], "InitialMirror", ";UNUSED_PARAMETER");  // This parameter is no longer used
            migrated = migrated | replaceString(configLines[i], ";InitialMirror", ";UNUSED_PARAMETER"); // This parameter is no longer used
            migrated = migrated | replaceString(configLines[i], "FlipImageSize", ";UNUSED_PARAMETER");  // This parameter is no longer used
            migrated = migrated | replaceString(configLines[i], ";FlipImageSize", ";UNUSED_PARAMETER"); // This parameter is no longer used
        }
        else if (section == "[Digits]")
        {
            migrated = migrated | replaceString(configLines[i], "LogImageLocation", "ROIImagesLocation");
            migrated = migrated | replaceString(configLines[i], "LogfileRetentionInDays", "ROIImagesRetention");
        }
        else if (section == "[Analog]")
        {
            migrated = migrated | replaceString(configLines[i], "LogImageLocation", "ROIImagesLocation");
            migrated = migrated | replaceString(configLines[i], "LogfileRetentionInDays", "ROIImagesRetention");
            migrated = migrated | replaceString(configLines[i], "ExtendedResolution", ";UNUSED_PARAMETER"); // This parameter is no longer used
        }
        else if (section == "[PostProcessing]")
        {
            /* AllowNegativeRates has a <NUMBER> as prefix! */
            if (isInString(configLines[i], "AllowNegativeRates") && isInString(configLines[i], ";"))
            {                                                                         // It is the parameter "AllowNegativeRates" and it is commented out
                migrated = migrated | replaceString(configLines[i], "true", "false"); // Set it to its default value
                migrated = migrated | replaceString(configLines[i], ";", "");         // Enable it
            }
            /* IgnoreLeadingNaN has a <NUMBER> as prefix! */
            else if (isInString(configLines[i], "IgnoreLeadingNaN") && isInString(configLines[i], ";"))
            {                                                                         // It is the parameter "IgnoreLeadingNaN" and it is commented out
                migrated = migrated | replaceString(configLines[i], "true", "false"); // Set it to its default value
                migrated = migrated | replaceString(configLines[i], ";", "");         // Enable it
            }
            /* ExtendedResolution has a <NUMBER> as prefix! */
            else if (isInString(configLines[i], "ExtendedResolution") && isInString(configLines[i], ";"))
            {                                                                         // It is the parameter "ExtendedResolution" and it is commented out
                migrated = migrated | replaceString(configLines[i], "true", "false"); // Set it to its default value
                migrated = migrated | replaceString(configLines[i], ";", "");         // Enable it
            }
            else
            {
                migrated = migrated | replaceString(configLines[i], ";PreValueUse = true", ";PreValueUse = false"); // Set it to its default value
                migrated = migrated | replaceString(configLines[i], ";PreValueUse", "PreValueUse");                 // Enable it

                migrated = migrated | replaceString(configLines[i], ";ErrorMessage = true", ";ErrorMessage = false"); // Set it to its default value
                migrated = migrated | replaceString(configLines[i], ";ErrorMessage", "ErrorMessage");                 // Enable it

                migrated = migrated | replaceString(configLines[i], ";CheckDigitIncreaseConsistency = true", ";CheckDigitIncreaseConsistency = false"); // Set it to its default value
                migrated = migrated | replaceString(configLines[i], ";CheckDigitIncreaseConsistency", "CheckDigitIncreaseConsistency");                 // Enable it
            }
        }
        else if (section == "[MQTT]")
        {
            migrated = migrated | replaceString(configLines[i], "SetRetainFlag", "RetainMessages");                   // First rename it, enable it with its default value
            migrated = migrated | replaceString(configLines[i], ";RetainMessages = true", ";RetainMessages = false"); // Set it to its default value
            migrated = migrated | replaceString(configLines[i], ";RetainMessages", "RetainMessages");                 // Enable it

            migrated = migrated | replaceString(configLines[i], ";HomeassistantDiscovery = true", ";HomeassistantDiscovery = false"); // Set it to its default value
            migrated = migrated | replaceString(configLines[i], ";HomeassistantDiscovery", "HomeassistantDiscovery");                 // Enable it

            if (configLines[i].rfind("Topic", 0) != std::string::npos) // only if string starts with "Topic" (Was the naming in very old version)
            {
                migrated = migrated | replaceString(configLines[i], "Topic", "MainTopic");
            }
        }
        else if (section == "[InfluxDB]")
        {
            /* Fieldname has a <NUMBER> as prefix! */
            if (isInString(configLines[i], "Fieldname"))
            {
                // It is the parameter "Fieldname"
                migrated = migrated | replaceString(configLines[i], "Fieldname", "Field"); // Rename it to Field
            }
        }
        else if (section == "[InfluxDBv2]")
        {
            /* Fieldname has a <NUMBER> as prefix! */
            if (isInString(configLines[i], "Fieldname"))
            {
                // It is the parameter "Fieldname"
                migrated = migrated | replaceString(configLines[i], "Fieldname", "Field"); // Rename it to Field
            }
            /* Database got renamed to Bucket! */
            else if (isInString(configLines[i], "Database"))
            {
                // It is the parameter "Database"
                migrated = migrated | replaceString(configLines[i], "Database", "Bucket"); // Rename it to Bucket
            }
        }
        else if (section == "[GPIO]")
        {
        }
        else if (section == "[DataLogging]")
        {
            migrated = migrated | replaceString(configLines[i], "DataLogRetentionInDays", "DataFilesRetention");
            /* DataLogActive is true by default! */
            migrated = migrated | replaceString(configLines[i], ";DataLogActive = false", ";DataLogActive = true"); // Set it to its default value
            migrated = migrated | replaceString(configLines[i], ";DataLogActive", "DataLogActive");                 // Enable it
        }
        else if (section == "[AutoTimer]")
        {
            migrated = migrated | replaceString(configLines[i], "Intervall", "Interval");
            migrated = migrated | replaceString(configLines[i], ";AutoStart = true", ";AutoStart = false"); // Set it to its default value
            migrated = migrated | replaceString(configLines[i], ";AutoStart", "AutoStart");                 // Enable it
        }
        else if (section == "[Debug]")
        {
            migrated = migrated | replaceString(configLines[i], "Logfile ", "LogLevel "); // Whitespace needed so it does not match `LogfileRetentionInDays`
            /* LogLevel (resp. LogFile) was originally a boolean, but we switched it to an int
             * For both cases (true/false), we set it to level 2 (WARNING) */
            migrated = migrated | replaceString(configLines[i], "LogLevel = true", "LogLevel = 2");
            migrated = migrated | replaceString(configLines[i], "LogLevel = false", "LogLevel = 2");
            migrated = migrated | replaceString(configLines[i], "LogfileRetentionInDays", "LogfilesRetention");
        }
        else if (section == "[System]")
        {
            migrated = migrated | replaceString(configLines[i], "RSSIThreashold", "RSSIThreshold");
            migrated = migrated | replaceString(configLines[i], "AutoAdjustSummertime", ";UNUSED_PARAMETER"); // This parameter is no longer used

            migrated = migrated | replaceString(configLines[i], ";SetupMode = true", ";SetupMode = false"); // Set it to its default value
            migrated = migrated | replaceString(configLines[i], ";SetupMode", "SetupMode");                 // Enable it
        }
    }

    if (migrated)
    {
        // At least one replacement happened
        if (!RenameFile(CONFIG_FILE, CONFIG_FILE_BACKUP))
        {
            LogFile.WriteToFile(ESP_LOG_ERROR, TAG, "Failed to create backup of Config file!");
            return;
        }

        FILE *pfile = fopen(CONFIG_FILE, "w");

        for (int i = 0; i < configLines.size(); i++)
        {
            if (!isInString(configLines[i], ";UNUSED_PARAMETER"))
            {
                fwrite(configLines[i].c_str(), configLines[i].length(), 1, pfile);
                fwrite("\n", 1, 1, pfile);
            }
        }

        fclose(pfile);
        LogFile.WriteToFile(ESP_LOG_INFO, TAG, "Config file migrated. Saved backup to " + string(CONFIG_FILE_BACKUP));
    }
}

@SybexX
Copy link
Collaborator

SybexX commented Oct 1, 2024

At the end, the ;UNUSED_PARAMETERs are filtered out and not saved in the config.ini.

@caco3
Copy link
Collaborator Author

caco3 commented Oct 1, 2024

The tricky part is to make sure the migration happens only once.
I have now found a simple solution by just checking the Brightness parameter:
image

@caco3 caco3 marked this pull request as ready for review October 1, 2024 22:04
@SybexX
Copy link
Collaborator

SybexX commented Oct 1, 2024

However, if this parameter does not exist in the config.ini, all others will be ignored^^

@caco3
Copy link
Collaborator Author

caco3 commented Oct 1, 2024

I do not want to touch the older migration pars. They are working and I do not want to invest time to test them once more.
But I like your handling of the unused parameter and added it to the branch.

@caco3
Copy link
Collaborator Author

caco3 commented Oct 1, 2024

However, if this parameter does not exist in the config.ini, all others will be ignored^^

Yes, thats true but very unlikely.
And even if so, nothing would break, just the cam settings get lost.
Most users anyways just use the UI and then they have them all.
And those who edit it manually know what they do.

@SybexX
Copy link
Collaborator

SybexX commented Oct 1, 2024

I tested my approach with the config.ini from version 15.7.0 and there were no problems with it.
The “else if()” makes the code more efficient and smaller when compiling

@caco3
Copy link
Collaborator Author

caco3 commented Oct 1, 2024

I am happy to use your approach, but please provide it as an alternative Pull request
😴

@SybexX
Copy link
Collaborator

SybexX commented Oct 1, 2024

with the config.ini from 15.7.0

Unbenannt

@SybexX
Copy link
Collaborator

SybexX commented Oct 1, 2024

this comes with my approach:
Unbenannt

I've always tested during normal operation, but this time I went through the setup and something is wrong with that^^

Supplement:
Problem is fixed (c708e23), but your approach only migrate Brightness.
Since the migrateConfiguration() processes the config.ini line by line and brightness is at the top, everything that follows is ignored.

@caco3
Copy link
Collaborator Author

caco3 commented Oct 2, 2024

Since the migrateConfiguration() processes the config.ini line by line and brightness is at the top, everything that follows is ignored.

Yes, you are right, my approach does not work.
Please create a branch/PR with your approach.

@caco3 caco3 closed this Oct 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

16.0.0-RC1: Zoom enabled is discarded after upgrade
2 participants