Using Python to Restore JIRA Custom Field Values from Issue History

Consider the following scenario: you move a bunch of issues from one type to another, or from one project to another. You don’t pay much attention and just confirm everything. Only then you find out some custom fields are gone and the data is lost – since the field context does not include the new issue type or project.

Don’t panic! no need to rush to restore the entire instance database from backup! The data is still visible in the issue’s History tab. All you need to do is fix the context of the fields and then restore it from the issue history. In this post I share some Python code (using python-jira module) to help you achieve that.

Step 1: Fix the field’s context

This better be done manually and carefully, if your policy is to maintain strict field configuration (which in general is a good idea since it improves Jira performance).

Go to administration > issues > custom fields and for each “lost” field which you wish to restore, click the cog icon > ‘Configure’ and then ‘Edit Configuration’. Set the proper issue types and projects.

Step 2: Prepare the restoration script

Now let’s prepare a python script to get a single issue key and collects the lost data from the issue history

Step 2a – Connect to JIRA and obtain the issue

# Connect to JIRA
from jira.client import JIRA
jira = JIRA({'server': jira_server_url}, basic_auth=(username, password)

# create mapping for custom field names and types
name_map = dict()
type_map = dict()
for field in jira.fields():
    if field['custom']:
        fname = get_current_field_name(field['name'])
        name_map[fname] = field['id']
        type_map[fname] = field['schema']['type']

# get the issue object - note the 'changelog' setting
issue = jira.issue(issue_key, expand='changelog')

Step 2b – Look for the lost fields in the history and get their original values

# get the values of the lost fields from the history entry where they got nullified
lost_fields = ['field 1', 'field 2']
original_value = dict()
for history in issue.changelog.histories:
    for item in history.items:
        field_name = item.field
        if field_name in lost_fields and not item.toString:
            # get the previous field value - in two forms, see below
            field_value_str = getattr(item, "fromString")  # string value
            field_value = getattr(item, "from")  # raw value (empty for string-based fields)
            field_type = type_map[field_name]
            if field_type == 'datetime':  # need to add milliseconds
                original_value[field_name] = field_value.replace('+', '.000+')
            elif field_type == 'user':  # username
                original_value[field_name] = field_value
            elif field_type == 'group':  # appears as array but we need it as string
                original_value[field_name] = field_value_str.replace('[', '').replace(']', '')
            elif field_type == 'array' and field_value:  # convert to python array - not using "eval" to avoid weird/malicious strings.
                original_value[field_name] = field_value_str.split(',')
            else:
                original_value[field_name] = field_value_str
               

Now debug/test the script on different issues to make sure it can handle the various fields and values you have. If you encounter an exception – most likely the field configuration is not correct, so double-check what you did in step 1.

Step 3 – Add code to restore the collected data

Go over the list of values we collected and update the issue with them. We need to update differently based on the field items, Jira’s REST API is anything but unified… some useful examples can be found here.

Note – we do not update fields which are not empty (e.g. someone re-entered them manually).

fields_to_update = dict()
for field_name in original_value.keys():
    field_real_name = name_map[field_name]
    field_type = type_map[field_name]
    # don't update a field which now has a value
    curr_value = getattr(issue.fields, field_real_name)
    if curr_value is None:
        value = values[field_name]
        if field_type == 'number':
            fields_to_update[field_real_name] = int(value)
        elif field_type in ['string', 'datetime']:
            fields_to_update[field_real_name] = value
        elif field_type in ['user', 'group', 'version']:
            fields_to_update[field_real_name] = {'name': value}
        elif field_type in ['array']:
            # for multi-version pickers, need to use [{'name': x} for x in value] 
            fields_to_update[field_real_name] = [{'value': x} for x in value]
        else:
            fields_to_update[field_real_name] = {'value': value}
        # useful debug print if needed. Helps comparing the outcome against the official REST API.
        # print("Adding {}({},{}) -> {}".format(field_name, field_real_name, field_type, fields_to_update[field_real_name]))
# now update the issue
if fields_to_update:
    try:
        print("Updating issue " + issue.key)
        issue.update(fields=fields_to_update)
    except Exception as e:
        print("{}: FAILED: {}".format(issue.key, e))
else:
    print("{}: Nothing to update".format(issue.key))

That’s it! Run it on the affected issues and everything’s back to normal.

Tip – if you’re going to run this script on a bunch of issues, I highly recommend to disable email notifications in your Jira instance beforehand 🙂

1 thought on “Using Python to Restore JIRA Custom Field Values from Issue History

  1. Nice !

    On Sun, Jan 10, 2021 at 12:34 AM Adventures in SCM wrote:

    > yossiz74 posted: ” Consider the following scenario: you move a bunch of > issues from one type to another, or from one project to another. You don’t > pay much attention and just confirm everything. Only then you find out some > custom fields are gone and the data is lost – sinc” >

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s