V3 Migration Guide
Many backward breaking changes occurred in the EdgeX 3.0 (Minnesota) release which may require some migration depending on your use case.
This section describes how to migrate from V2 to V3 at a high level and refers the reader to the appropriate detail documents. The areas to consider for migrating are:
- Customized Configuration
- Custom Compose File
- Command Line Options
- Custom Device Service
- Custom Device Profile
- Custom Pre-Defined Device
- Custom Applications Service
Service configuration is one of the big changes for EdgeX V3
If you have customized any EdgeX service's configuration (core, support, device, etc.) via the Configuration Provider (Consul), those customization will need to be re-applied to those services' configuration or the common configuration in the Configuration Provider once the V3 versions have started and pushed their configuration into the Configuration Provider. The V3 services now use
v3 in the Configuration Provider path rather than
2.0 . The folder structure in the Configuration Provider has been flattened so all services are at the same level. See the Configuration File section below for details on migrating configuration.
Example Configuration Provider paths for V3
If the Configuration Provider data is not cleared prior to running the V3 services, the V2 configuration will remain and be taking up useful memory. The configuration data in the Configuration Provider can be cleared by deleting the
.../edgex/ node with the curl command below prior to starting EdgeX 3.0.
curl --request DELETE http://localhost:8500/v1/kv/edgex?recurse=true`
If you have customized the service configuration files for any EdgeX service (core, support, device, etc.) that configuration will need to be migrated.
The biggest two changes to the service configuration files are:
- File format has changed to YAML
- Settings that are common have been removed from each service's local private configuration file
See V3 Migration of Common Configuration for the details on migrating configuration common to all EdgeX services.
The tool here can be used to convert your customized service configuration file from TOML to YAML. This should be done once all the common configuration has been removed.
The following are where you can find the configuration migration specifics for individual EdgeX services
- Core Data
- Core Metadata
- Core Command
- Support Notifications
- Support Scheduler
- Application Services
- Device Services (common)
- Device MQTT
- Device ONVIF Camera
- Device USB Camera
Customized Environment Overrides
If you have custom environment overrides for configuration impacted by the V3 changes you will also need to migrate your overrides to use the new name or value depending on what has changed. Refer to the links above and/or below for details for migration of common and/or the service specific configuration to determine if your overrides require migrating.
When using the Configuration Provider, the environment overrides for common configuration are applied to the core-common-config-bootstrapper service. They no longer work when applied to the individual services as the common configuration setting no longer exist in the private configuration.
Custom Compose File
The compose files for V3 have many changes from their V2 counter parts. If you have customized a V2 compose file to add additional services and/or add or modify configuration overrides, it is highly recommended that you start with the appropriate V3 compose file and re-add your customizations. It is very likely that the sections for your additional services will need to be migrated to have the proper environment overrides. Best approach is to use one of the V3 service sections that closest matches your service as a template.
The latest V3 compose files can be found here: Compose Files
If the additional service(s) in your custom compose file are EdgeX released device or app services, it is highly recommended that you use the Compose Builder to regenerate your custom compose file.
The latest V3 Compose Builder can be found here: Compose Builder Readme
Command Line Options
The following command-line options and corresponding environment variables have be renamed for consistency
-c/--confdiris replaced by
EDGEX_CONF_DIRenvironment variable is replaced by
-f/--fileis replaced by
EDGEX_CONFIG_FILEhas not changed
-cp/ --configProviderhas not changed
EDGEX_CONFIGURATION_PROVIDERenvironment variable is replaced by
If your solution uses any of the renamed options or environment variables you will need to make the appropriate changes to use the new names.
There currently is no migration path for the data stored in the database. If possible, the database should be cleared prior to starting V3 EdgeX. This will allow the database to be V3 compliant from the start. See Clearing Redis Database section below for details on how to clear the Redis database.
The following sections describe what you need to be aware for the different services that create data in the database.
The Event/Reading data stored by Core Data is considered transient and of little value once it has become old. The V3 versions of these data collections have minimal changes from their V2 counter parts.
- Add Event
To identify which device service generating the new event, POST endpoint is now changed from
See Core Data API Reference for complete details.
tagsfield in reading.
The field that has changed in V3 is the
apiVersion which is now set to
Most of the data stored by Core Metadata will be recreated when the V3 versions of the Device Services start-up. The statically declared devices will automatically be created and device discovery will find and add existing devices. Any device profiles, devices, provision watchers created manually via the V2 REST APIs will have to be recreated using the V3 REST API. Any manually-applied
AdministrativeState settings will also need to be re-applied.
Add/ Update/ Get device
UpdateLastConnectedfrom device model
- Updated ProtocolProperties to have typed value
Add/ Update/ Get deviceprofile
optionalfield in ResourceProperties
- Updated the data type of
- Changed the response format from TOML to YAML
Add/ Get/ Update ProvisionWatcher
- Allowed empty string profile name when adding or updating the ProvisionWatcher
- The ProvisionWatcher DTO is restructured by moving the Device related fields into a new object field,
DiscoveredDevice; such as
- Added a new properties field in the
DiscoveredDeviceobject to allow any additional or customized data.
- ProvisionWatcher contains its own
adminStatenow. The Device
adminStateis moved into the
Add/ Update Device
- Removed the boolean field
notifywhich is never used
- Added the new field
- Removed the boolean field
See Core Metadata API Reference for complete details.
- Get Command
ds-returneventto use bool value,
false, instead of
See Core Command API Reference for complete details.
Subscriptions created via the V2 REST API will have to be recreated using the V3 REST API. The
Transmissioncollections will be empty until new notifications are sent using EdgeX V3
authmethodto support-scheduler actions DTO, which indicates how to authenticate the outbound URL. Use
NONEwhen running in non-secure mode and
JWTwhen running in secure mode.
See Support Scheduler API Reference for complete details.
The statically declared
IntervalAction will be created automatically. Any
IntervalAction created via the V2 REST API will have to be recreated using the V3 REST API. If you have created a custom configuration with additional statically declared
IntervalActions see the Configuration File section under Customized Configuration below.
Application services use the database only when the Store and Forward capability is enabled. If you do not use this capability you can skip this section. This data collection only has data when that data could not be exported. It is recommended not to upgrade to V3 while the Store and Forward data collection is not empty or you are certain the data is no longer needed. You can determine if the Store and Forward data collection is empty by setting the Application Service's log level to
DEBUG and look for the following message which is logged every
msg=" 0 stored data items found for retrying"
Clearing Redis Database
When running EdgeX in Docker the simplest way to clear the database is to remove the
db-data volume after stopping the V2 EdgeX services.
docker compose -f <compose-file> down
docker volume rm $(docker volume ls -q | grep db-data)
Now when the V3 EdgeX services are started the database will be cleared of the old v2 data.
Because there are no tools to migrate EdgeX configuration and database, it's not possible to update the edgexfoundry snap from a V2 version to a V3 version. You must remove the V2 snap first, and then install a V3 version of the snap (available from the 3.0 track in the Snap Store). This will result in starting fresh with EdgeX V3 and all V2 data removed.
If you are running EdgeX locally, i.e. not in Docker or snaps and in non-secure mode you can use the Redis CLI to clear the database. The CLI would have been installed when you installed Redis locally. Run the following command to clear the database:
This will not work if running EdgeX in running in secure mode since you will not have the random generated Redis password unless you created an Admin password when you installed Redis.
Custom Device Service
If you have custom Device Services they will need to be migrated to the V3 version of the Device SDK. See Device Service V3 Migration Guide for complete details.
Custom Device Profile
If you have custom V2 Device Profile(s) for one of the EdgeX Device Services they will need to be migrated to the V3 version of Device Profiles. See Device Service V3 Migration Guide for complete details.
Custom Pre-Defined Device
If you have custom V2 Pre-Defined Device(s) for one of the EdgeX Device Services they will need to be migrated to the V3 version of Pre-Defined Devices. See Device Service V3 Migration Guide for complete details.
Custom Applications Service
If you have custom Application Services they will need to be migrated to the V3 version of the App Functions SDK. See Application Services V3 Migration Guide for complete details.
If you have an add-on services running in secure mode you will need to use the new names of the environment variables in EdgeX V3. See Security Services V3 Migration Guide for more details.
API Gateway configuration
The API gateway has changed in EdgeX V3. See Security Services V3 Migration Guide for more details.
Authenticated REST APIs
When security is enable, all V3 EdgeX services REST APIs require a JWT authorization token. See Security Services V3 Migration Guide for more details.
None Secure Mode
If running EdgeX in none secure mode and you have rules with
rest action that reference an EdgeX service the endpoint API version will need to be changed from v2 to V3
Example migration of
rest action with EdgeX endpoint
If running EdgeX in secure mode and you have rules with
rest action that reference an EdgeX Core Command you will need to convert the rule to use Command via External MQTT. See eKuiper documentation here for more details. This is due to the new microservice authorization on all EdgeX services' endpoints requiring a JWT token which eKuiper doesn't have.
This approach requires an external MQTT broker to send the command requests. The default EdgeX compose files do not include a MQTT Broker. This broker is supposed to be external to EdgeX.