Export data from SharePoint Online List
Exporting SharePoint list to Excel or CVS is one requirement that have been raised again and again. We approach to solution usually by opening Excel and importing data from SharePoint Online. We will go through details later but first; this approach has one huge problem and huge means huge.
Let us share a screenshot.
We have started to get data from SharePoint Online list and after about 105 minutes, 7815 rows loaded out of 32932. Do math and you will have idea how much time it will take. MS documentations states that we can have 30 million items in a list and 32932 rows are 0.1 percent of limit.
To export list with “large data”, we return to our friend PowerShell and CSOM. When list data is “huge”, we will be using PowerShell and CSOM and when list data is not “huge”, we prefer importing data to Excel.
Let’s start with importing SharePoint Online list data to Excel. Start Excel and go to select “Data” tab. Click on “Get Data” -> “From Online Services” -> “From SharePoint Online List”.
Pop up will appear with heading “SharePoint lists”. Don’t get fool with this heading and enter URL of SharePoint Online site, not URL of SharePoint Online list.
Pop up will appear connecting
List of all lists will appear
Select desired list and data will appear in preview window
Click on “Load” and pop up will appear
Eventually, you will be able to see data from SharePoint Online list. Don’t be afraid looking at all jargon that appear in Excel. Go to “Query” tab under “Query Tools” and click on “Edit”.
“Query Editor” window will appear. Scroll horizontally and find column “FieldValuesAsText”.
Make sure “FieldValuesAsText” is selected and click on “Remove Columns” -> “Remove Other Columnes”. This will remove all columns from “Query Editor”.
Once all columns are removed, click on right corner on “FieldValuesAsText” and pop will appear.
Select columns that are required, Excel will take time to load and you will have all columns from SharePoint List along with data. Rename columns and click “Save & Load”. This will take time as Excel will update itself based on changes made and we have imported data from SharePoint Online list to Excel.
One last detail is about “Query Settings” window. This window, display all the changes performed in sequential order and can undo any change.
This approach works fine when we are working with small data size but takes a lot time with each step added in Query Settings. One way is to don’t add any step in Query Settings and try to with work with data populated in Excel from SharePoint Online list in first place. Other way is to write PowerShell script and export data from SharePoint Online list to CSV or Excel. We will be using CSV format for simplification.
First step is make sure that Microsoft.SharePoint.Client.dll and Microsoft.SharePoint.Client.Runtime.dll at following location
Please specify values of following parameters and we will get context of SharePoint Online site
We will be using following CAML query which will have OOTB columns. We can add as many columns as desired
Accessing SharePoint Online site and list in script below
$secpasswd = ConvertTo-SecureString $userPassword -AsPlainText -Force
$context = New-Object Microsoft.SharePoint.Client.ClientContext($siteURL)
$context.Credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($UserName, $secpasswd)
$list = $context.Web.Lists.GetByTitle($listtitle)
$context.Load($list)
$context.ExecuteQuery()
Below is important part of script. First is $position variable which will store the current position and $itemsinfo is array of PSObject
$position = $null
$itemsinfo = @()
We will be using Do Until loop and will be terminating when $position variable is null and adding data in $itemsinfo
Do
{
$camlQuery = New-Object Microsoft.SharePoint.Client.CamlQuery
$camlQuery.ListItemCollectionPosition = $position
$currentCollection = $list.GetItems($qCommand)
$context.Load($currentCollection)
$context.ExecuteQuery()
$position = $currentCollection.ListItemCollectionPosition
foreach($listitem in $currentCollection)
{
try
{
$fieldvalue = @{
Created = [System.TimeZoneInfo]::ConvertTimeFromUtc($listitem["Created"], $TZ)
Modified = [System.TimeZoneInfo]::ConvertTimeFromUtc($listitem["Modified"], $TZ)
}
}catch
{
Write-Host $_
}
$itemsinfo += New-Object psobject -Property $fieldvalue
}
}
Until($position -eq $null)
Last step is to create CSV file
$itemsinfo | Select-Object Created, Modified | export-csv "C:\DataFromSharePointOnlineList.csv" -NoTypeInformation
Full script is as under
Add-Type -Path "C:\Program Files\Common Files\microsoft shared\Web Server Exten-sions\15\ISAPI\Microsoft.SharePoint.Client.dll" -ErrorAction Stop
Add-Type -Path "C:\Program Files\Common Files\microsoft shared\Web Server Exten-sions\15\ISAPI\Microsoft.SharePoint.Client.Runtime.dll" -ErrorAction Stop
$strCurrentTimeZone = (Get-WmiObject win32_timezone).StandardName
$TZ = [System.TimeZoneInfo]::FindSystemTimeZoneById($strCurrentTimeZone)
$username = "XXXXXX"
$userPassword = "XXXXXXXXX"
$siteURL = "XXXXXXX"
$listtitle = "XXXXXX"
$qCommand = @"
<View Scope="RecursiveAll">
<Query>
<OrderBy Override='True'><FieldRef Name='Modified' /></OrderBy>
</Query>
<ViewFields>
<FieldRef Name='Modified' /><FieldRef Name='Created' />
</ViewFields>
<RowLimit Paged="TRUE">5000</RowLimit>
</View>
"@
$secpasswd = ConvertTo-SecureString $userPassword -AsPlainText -Force
$context = New-Object Microsoft.SharePoint.Client.ClientContext($siteURL)
$context.Credentials = New-Object Mi-crosoft.SharePoint.Client.SharePointOnlineCredentials($UserName, $secpasswd)
$list = $context.Web.Lists.GetByTitle($listtitle)
$context.Load($list)
$context.ExecuteQuery()
$position = $null
$itemsinfo = @()
Do
{
$camlQuery = New-Object Microsoft.SharePoint.Client.CamlQuery
$camlQuery.ListItemCollectionPosition = $position
$currentCollection = $list.GetItems($qCommand)
$context.Load($currentCollection)
$context.ExecuteQuery()
$position = $currentCollection.ListItemCollectionPosition
foreach($listitem in $currentCollection)
{
try
{
$fieldvalue = @{
Created = [System.TimeZoneInfo]::ConvertTimeFromUtc($listitem["Created"], $TZ)
Modified = [Sys-tem.TimeZoneInfo]::ConvertTimeFromUtc($listitem["Modified"], $TZ)
}
}catch
{
Write-Host $_
}
$itemsinfo += New-Object psobject -Property $fieldvalue
}
}
Until($position -eq $null)
$itemsinfo | Select-Object Created, Modified | export-csv "C:\DataFromSharePointOnlineList.csv" -NoTypeInformation
One last workaround if want to create Excel, following code will generate Excel from CSV
$excel = New-Object -ComObject excel.application
$excel.visible=$false
$excel.DisplayAlerts = $false
$reportOut = $excel.Workbooks.Add()
$wb = $excel.WorkBooks.Open("C:\DataFromSharePointOnlineList.csv")
$wb.Worksheets.Item(1).Name = "DataFromSharePointOnlineList"
$wb.Worksheets.Copy($reportOut.WorkSheets.Item(1))
$wb.Close(0)
$reportOut.worksheets.item("Sheet1").Delete()
$strdate = get-date
$filename = "C:\DataFromSharePointOnlineList.xlsx"
$reportOut.SaveAs($filename,[Microsoft.Office.Interop.Excel.XlFileFormat]::xlOpenXMLWorkbook)
$reportOut.Close(0)
$excel.Quit()