Streamlining Umbraco 8 Content Imports and Copy Language Variant Functionality
There are 2 problems that we recently faced dealing with content import and language variant management, and we thought this would be a great opportunity to discuss it.
First, there is the issue of content import. We’ve all been there. We have to import a lot of various content into our Umbraco system. Fortunately, there are some great plugins for doing this, like the CMS Import plugin. But what if you want total control over the import process? What if you want very specific results and know exactly what you want?
You can code the import process yourself! Umbraco provides a slew of API methods that you can use to import your data into the CMS exactly how you want them. While it was written for Umbraco 8, this code could be easily modified to work with Umbraco 9 and 10, as most of the API calls are the same.
Second, there is the issue of content variants, and Umbraco’s lack of a “copy variant” option. This can quickly turn into a significant issue when you start dealing with a lot of multilingual content. Fortunately, the solution for this was very easy – just create a new copy variant feature as an app plugin.
Let’s dive into this in more detail.
The Problems
- Multiple content import sources. Our project was a massive multilingual project. We had both general data in GatherContent that would be translated and imported and specific content fields within the Umbraco system that needed to be exported for translation, then re-imported back in.
- We were using multilanguage variants with the same basic language (ex. en-US vs. en-CA), but Umbraco doesn't allow copying variant values from one to the other, making content entry very tedious.
The Solutions
First: Umbraco API to the Rescue!
Our content import sources could be broken into these categories:
- GatherContent
- Umbraco Dictionary
- Umbraco Content
The idea is that we can build endpoints to do what we want on a surface controller and execute the endpoints as we need to via the URL. We made content manipulation CSV-based and made use of the excellent CsvHelper package.
The first thing I'll say is you should be concerned about security implications or even unintended executions. In our implementation, we restricted access to these surface controller APIs to logged-in users and restricted execution to admins. We also restricted execution of endpoints that update content to a local domain only. That way these couldn't be executed in the production environment and overwrite existing language content. Note that we could have implemented UmbracoAuthorizedApiController instead, which provides baked-in backoffice security checks. Also, the content import code should not exist in the production environment; only in a local or development environment.
An Umbraco plugin already exists for GatherContent. Still, because we knew exactly what functionality we wanted, we decided to write a custom surface controller which did exactly what we wanted. We could export a CSV from GatherContent and execute a surface controller to import in the data exactly in the way that we wanted. We were able to handle different content type imports differently, even initializing a grid for certain content types.
Our generated endpoints:
Import from GatherContent CSV:
/umbraco/surface/mymigrationhelper/UpdateContentVariantsWithGatherContentJson?filename=sample.csv&languageiso=en-us
For the Umbraco dictionary, this was rather easy to code. We wrote a surface controller for exporting the data out as a CSV in one of 2 ways: vertical or horizontal. We executed this and sent it to the translators. We then created a surface controller for importing the new language CSV back in. When the translators returned the new data, we could import it back in. You can actually reuse this code without much modification as the data structure should be consistent and the dictionary data structure is static.
Export to CSV:
/umbraco/surface/mymigrationhelper/GetDictionaryVariantContent
Import from CSV:
/umbraco/surface/mymigrationhelper/UpdateTranslationDictionaryVariantsJson?filename=samplevertical.csv&style=vertical
For the Umbraco content, this got very complicated. We needed to export every content field. What we decided on was an endpoint that looped through properties systematically and created a CSV for export. We could then execute an importer that imported it back in. This would have worked, but since we were using grid editors on this site (including Doc Type Grid Editor), we realized this format wasn't that realistic for content editors to edit.
We decided on exporting the CSV data and then manually cutting it down to more basic fields to be easier for content translators. When it came time to import this data back in, we could specifically handle each column property field as needed.
Export to CSV:
/umbraco/surface/mymigrationhelper/GetNodeVariantContent/?doctype=[doctypename]
Import from CSV:
/umbraco/surface/mymigrationhelper/UpdateContentVariantsJson?filename=sample.csv
I won’t go into detail for the source code in this post, but you can download it and add it to your project if you want to play around with it. In all, this wasn’t a perfect solution and required some manual work to look at the data and tweak it in the CMS, but it was far better than a fully manual process and saved us a lot of time.
Download the CSV Import Source Code
Creating an App Plugin to Copy Variants
For our project, we quickly realized that we were using variants of the English language where many variants (en-GB, en-CA, en-SG, etc.) should be populated by the primary language. Unfortunately, out of the box, Umbraco does not provide a way for copying variant values within a node, so we decided to create an app plugin which allows an editor to do so.
This was fun to build, as it would be very useful not only for the initial content import, but also for the client going forward. We decided to add a new Umbraco content menu option which allows the content editor to copy all of a node's property field values from one variant to another. This ended up being an incredibly useful tool and saved us a lot of time.
We simply made use of an Umbraco Authorized API Controller for the code. The important part looks like this. First is the backoffice endpoint that is hit via AJAX in our plugin:
[HttpGet]
public ApiResponseMessage<string> CopyNodeVariant(int nodeId, string sourceIso, string targetIso)
{
var contentService = Services.ContentService;
var node = contentService.GetById(nodeId);
return CopyNodeVariantLogic(node, sourceIso, targetIso);
}Then the CopyNodeVariantLogic code:
private ApiResponseMessage<string> CopyNodeVariantLogic(IContent node, string sourceIso, string targetIso)
{
sourceIso = sourceIso.ToLowerInvariant();
targetIso = targetIso.ToLowerInvariant();
ApiResponseMessage<string> result = new ApiResponseMessage<string>();
result.Messages = new List<string>();
var contentService = Services.ContentService;
//ok, read node
if (sourceIso != targetIso)
{
if (node != null)
{
if (node.AvailableCultures.Any(x => x.ToLowerInvariant() == sourceIso.ToLowerInvariant()))
{
try
{
//loop through all property variants of sourceIso, set targetiso
foreach (var myProperty in node.Properties)
{
if (myProperty.PropertyType.Variations.ToString() == "Nothing")
{
//do nothing
}
else
{
var sourceValue = myProperty.GetValue(sourceIso);
myProperty.SetValue(sourceValue, targetIso);
}
}
//save and publish
node.SetCultureName(node.Name, targetIso);
contentService.SaveAndPublish(node, targetIso, -1, false);
result.Success = true;
result.Data = "" + node.Id + " " + node.Name + " variant " + sourceIso + " to " + targetIso + " was successful.";
}
catch (Exception ex)
{
result.Messages.Add("Error: " + ex.Message);
}
}
else
{
result.Data = "Note, the node " + node.Id + " " + node.Name + " does not have the source culture content saved yet. Skipped.";
result.Messages.Add("Note, the node " + node.Id + " " + node.Name + " does not have the source culture content saved yet. Skipped.");
result.Success = true;
}
}
else
{
result.Messages.Add("Error, node could not be retrieved");
}
}
else
{
result.Messages.Add("Error, you must choose different source and target languages");
}
return result;
}As you can see, it really isn’t that complicated. This could be expanded to selectively updating specific fields, copying descendant node variant values, and more.
We just pair this back-end code with a front-end App_Plugin folder, View, CSS, and JS, and we are in business. Our app plugin can execute our endpoint via AJAX and return the result.
Download the Copy Variants Source Code
Please note that the source code is included for you as a developer to analyze and learn from. These are code snippets and not an entire solution. For this copy variants source code, there is no configuration required to get it working. Just copy the “Website Project” contents to your website project and the “Backend Project” contents into your backend (C# classes) project.
Summary
Because Umbraco offers endpoints that developers can use and offers API access to content, developers can create their own tools to make content import easier, data more consistent, and manual effort kept to a minimum.
I also learned a few things from this process:
- With content imports, first analyze the existing data structure from the Umbraco ContentService. You will need to match this when the data is imported in.
- With content import endpoints, always provide a ?previewOnly=true query string to allow you to see what would have happened if you had executed the import. This greatly reduces errors and limits the issues that you have.
- Always perform the first import on a single item. Don’t attempt to import in all content at once from fresh code. If your code is new, first preview it, then execute it on a single item. If that works, you can execute it for all content.