Skip to main content

Notifications

Power Automate - Building Flows
Answered

Migrating files - can't update metadata from source Lookup column in destination

(0) ShareShare
ReportReport
Posted on by 68
Because Microsoft have been switching some features in Classic SharePoint off, I want to migrate some large document libraries from Classic to new Modern sites. I would have thought this might be a fairly straightforward process ... and yes, I understand that the source library's Created Date/By and Modified Date/By values won't be ported across, but we can live with that.
 
From - /sites/eo-wordings-libray/WordingsLibrary(Archive)
To -     /sites/eo-wordings-library/WordingsLibrary(Archive)

[Note the subtle change in site name from the original, which was spelt incorrectly, and not by me!]

So, despite finding other more complex methods via Google, the simplest one has proved the most effective, though not completely effective. This is what I've got:



There's a filter on the Get Items function which limits the run to five items from one View of the library. This is to keep the run short for testing purposes.

The destination library is a clone of the source library (created by basing the new library in Modern on the old library in Classic). There are 56 columns in each library, excluding the mandatory Modified and Created columns.

There are three Lookup columns in each Library - Class (32 rows in Lookup list), LOB (126 rows in LookupList) and PublicationBrokers (17 rows in Lookup List).

When I run the Flow, the documents (PDFs and Word) are copied to the destination Library, along with almost all the metadata - except for the value for Class. I do not know why this is.

It can't be a limit on the number of items in the Lookup column, because LOB has more items, and those values copy across just fine.

The three Lookup columns are formatted identically, each looking up their values from the three separate Lookup Lists.

I tried adding an Update file properties function, and setting the value for Class Id (it didn't offer a field for Class value) to Class Id from Get Items but that didn't work. The value for Class still didn't transfer across.



But I did notice one curious thing. While digging around in the Run history, I had a look at what the Update file properties function was doing and noticed this:



The Class column output has no line for Value (though the ID number is correct), yet the LOB column output does.

I also wondered if there was some way of using the Send an HTTP request to SharePoint function as some postings in forums had referred to it as a possible solution.



I gave it a try but that didn't work either - though it's perfectly possible I made some dumb mistake in the syntax:
 
 
Sadly none of this has worked. The Class column in the destination library stubbornly refuses to update, and now I'm at a loss.

Updating the values manually for Class post-migration is out of the question as there are over 6000 documents in this one library and four other libraries totalling about the same number of documents between them - so I wouldn't relish adding Class values to 12,000 documents one at a time.

It's hard to believe there isn't some way of fixing this, I'm probably overlooking something really simple ... any help anyone here can give would be gratefully received.
 
I've tried to include all the useful information I can, but if additional info is needed, post a comment and I'll respond as fast as I can.
 
Thanks in advance,
Alan McK
https://thesharepointhive.blogspot.com/
 
  • AlanMcK Profile Picture
    AlanMcK 68 on at
    Migrating files - can't update metadata from source Lookup column in destination
    Thank you for your response, David_MA ...

    However,  the fact remains that - in this case - when copying values from a source list to a destination list I had to keep the same ID numbers, otherwise, the Lookup column values wouldn't be migrated from the source Library to the destination Library.

    And to your point that ID numbers increment by a value of 1 every time - that's not the case. Just this afternoon I watched the ID number in the destination list literally jump from 27 to 1000028 as I pasted in the next value. Not quite sure what was causing that, but I suspect it's a glitch in the matrix when the security token of the SharePoint page displaying the List times out. But maybe you know otherwise.

    Either way, I had to start again, then carefully add the values to the destination list in such a way that the same ID numbers were preserved. Otherwise the migration Flow just wouldn't work.

    The upshot is that I did what I did and my Flow finally worked, documents were migrated with the correct metadata. And if someone else finds themselves in a similar situation, well, now they know how to rectify it ...

    Alan McK
  • David_MA Profile Picture
    David_MA 8,364 on at
    Migrating files - can't update metadata from source Lookup column in destination
    AlanMcK, you may not already know this based on your last post, but the ID values in SharePoint are not "erratic." They are quite logical as the ID column is the primary key within every SharePoint list and document library. They start at 1 and increment by 1 every time a new item is added to the list or document library. Even if you delete an item in a list, the ID will never be used again.
     
    When you are copying items from one SharePoint site to another, you should not rely on these numbers being the same as it would be difficult to ensure they stay in sync since the value is generated at the time the item is created, and you cannot control the value.

    If you need to have the old values from your existing list, I would create a number column and set it to 0 decimal places. You may want to name it OriginalID and set it to require unique values. When you migrate your lists, populate the ID from the origin to this field in the destination. 
  • Verified answer
    AlanMcK Profile Picture
    AlanMcK 68 on at
    Migrating files - can't update metadata from source Lookup column in destination
    Thank you, ABC_123 ...

    You set me off down the right track. You're right. It was all to do with the Lookup Lists.

    But it wasn't to do with the structure of the Destination Lookup Lists or how they'd been created. It was all about how the Destination Lists were populated from the Source Lists.

    The issue had been staring me in the face the whole time. When the Flow copied values from the source Lookup Column to the destination Lookup Column, it used the ID number (not the Column Value). It turns out that the source Lookup List had erratic ID numbers. For example the source Class List started with ID=63, and the destination started with ID=1. So you can see why the Flow wasn't able to copy the Class Column value over to the Destination. Because the IDs didn't match. In fact, ID=63 didn't exist in the Destination List.



    Arrghhhh - SharePoint!

    It was very, very fiddly, trying to reconstruct the Lookup Lists in the destination site so that the ID numbers matched (it was something I was stuck with as I didn't create the original Libraries, and couldn't make any changes there). So after a few false starts, I just cautiously added the rows to the destination Lists, taking care not to let the security of the page time out (otherwise you suddenly start getting IDs like 10000026).

    Once the Lists were populated correctly, the Flow functioned exactly as expected.

    My next big task will be copy the files over in batches so that the Flow doesn't hiccup under the strain ... but, you know, 6000 documents and that!

    Thanks again, ABC_123 ... and I hope this tricky-challenge-explained helps someone.

    Alan McK
     
  • abc 123 Profile Picture
    abc 123 713 on at
    Migrating files - can't update metadata from source Lookup column in destination
    I'm guessing that there is something amiss with the lookup list in the new modern site. I'd try deleting the column from the library, and delete the list, then recreate the list and readd the column. Maybe even go so far as to create the new list as a New list, and not use the Copy from Existing option. 

Under review

Thank you for your reply! To ensure a great experience for everyone, your content is awaiting approval by our Community Managers. Please check back later.

Helpful resources

Quick Links

Kickstarter Events…

Register for Microsoft Kickstarter Events…

Tuesday Tip #12 Start your Super User…

Welcome to a brand new series, Tuesday Tips…

Tuesday Tip #13 Writing Effective Answers…

Welcome to a brand new series, Tuesday Tips…

Leaderboard

#1
WarrenBelz Profile Picture

WarrenBelz 144,858

#2
RandyHayes Profile Picture

RandyHayes 76,287

#3
Pstork1 Profile Picture

Pstork1 64,505

Leaderboard