Landmarks tps format

[EveTC]

Hi Steven (just me again! hehe),

I was just wondering if patternize accepts landmarks in a tps format? For example files created using the programme tpsUtil and tpsDig? This would save having to export the x&y coordinates seperately for hundreds of individuals as it is automatically saved into one file?

I hope you are well and look forward to hearing from you. Thank you, Eve

[StevenVB12]

Hi Eve,

I haven't been using tips myself. Could you share me an example with three images? I will try to add that option.

Cheers,

Steven

-------- Original message -------- From: EveTC [email protected] Date: 10/07/2019 08:12 (GMT-04:00) To: StevenVB12/patternize [email protected] Cc: Subscribed [email protected] Subject: [StevenVB12/patternize] Landmarks tps format (#15)

Hi Steven (just me again! hehe),

I was just wondering if patternize accepts landmarks in a tps format? For example files created using the programme tpsUtil? This would save having to export the x&y coordinates seperately for hundreds of individuals as it is automatically saved into one file?

I hope you are well and look forward to hearing from you. Thank you, Eve

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/StevenVB12/patternize/issues/15?email_source=notifications&email_token=ABQOC455YJGB5IE34JBJQ7LP6XG4BA5CNFSM4H7OVVJ2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4G6KYSDQ, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABQOC4Y2VKQPNRMJNP5OYDDP6XG4BANCNFSM4H7OVVJQ.

[EveTC]

Hi Steven,

Thank you so much for your quick reply.

Here is an example of one of my calibrated (using patternize :D) images that I want to set landmarks on

I make a tps file using tpsUtil and place 13 landmarks on each of these individuals using tpdDig following the tutorials (1 and 2) found here: [https://www.youtube.com/channel/UC2yXTBdhPgVL6H7hW8rMcFw]

which looks like this:

and produces a file with this format (copied it into a text editor as I cant upload a .tps file to GitHub for some reason): practice_etc.txt

Let me know if you need anything else. Thank you so much, Eve

[StevenVB12]

Hi Eve,

Thanks for the list. I added the option to use a tps landmark file to the makeList function (please reinstall patternize from github for this to work).

It works as follows:

landmarkList_tps <- makeList(IDlist, 'landmark', format = 'tps', tpsFile = 'tps_filename')

Please let me know if it works correctly. I think in your further commands you should set adjustCoords = FALSE for tps landmarks.

Cheers, Steven

[StevenVB12]

Also, note that your IDlist should have the same order as the samples in the tpsFile.

Steven

[EveTC]

Hi Steven,

Amazing, thank you so much! I will give it a go shortly. I realised I didnt set a scale on my example before (which I will be doing) which makes the file format as so: practice_scale_etc.txt Does this change what you have just done? So sorry if it does!

It is amazing how quick you always solve my queries - thank you so so much. Eve

[StevenVB12]

I don't think it should matter, but let me know if things don't work!

Steven

[EveTC]

Hi Steven,

So I have run it with the landmarks input as a tps format. Firstly with the scale and secondly without the scale in the tps.

The landmarkList_tps_scale <- makeList(IDlist, 'landmark', format='tps', tpsFile= './photos/GBS pops/tps_prac/practice_scale_etc.TPS') is accepted without errors.

However when I plot the summed rasters using: rasterList_lanK_scale <- patLanK(imageList, landmarkList_tps_scale, k=3, adjustCoords = FALSE, plot=T, resampleFactor = 10, removebgK = 220) summedRaster_lanK_scale <- sumRaster(rasterList_lanK_scale, IDlist, type = 'k') colfunc <- inferno(100) plotHeat(summedRaster_lanK_scale, IDlist, colpalette = colfunc, adjustCoords = FALSE)

It produces this with the scale in the landmarks folder:

It produces this without the scale:

Here is my script if you need: tps_landmark_test_script.txt

Have I made a silly mistake? I am just trying it now with landmarks set the original methos for these three individuals to see if it works.

Thanks, Eve

[EveTC]

I have just tried it with the normal method of importing landmarks and get the same output as above when I do not have the scale in the tps format. What have I done wrong?

Thanks, Eve

[StevenVB12]

Hi Eva,

Could you share me a couple of images with the landmarks and code for me to try? Maybe you can send them to my email vanbelleghemsteven at hotmail.com.

Steven

-------- Original message -------- From: EveTC [email protected] Date: 11/07/2019 08:20 (GMT-04:00) To: StevenVB12/patternize [email protected] Cc: "Steven M. Van Belleghem" [email protected], Comment [email protected] Subject: Re: [StevenVB12/patternize] Landmarks tps format (#15)

I have just tried it with the normal method of importing landmarks and get the same output as above when I do not have the scale in the tps format. What have I done wrong?

Thanks, Eve

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/StevenVB12/patternize/issues/15?email_source=notifications&email_token=ABQOC46C6UT6DDZGUXE7TQTP64QRFA5CNFSM4H7OVVJ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZWQTMA#issuecomment-510462384, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABQOC466VQX2KPTKYDCZ5XDP64QRFANCNFSM4H7OVVJQ.

[EveTC]

yes sure :) sending them your way now.

[StevenVB12]

Hi Eve,

I had to make a couple of improvements to allow for the correct plotting when using tps files (please reinstall).

In the process I also realized I could improve the k-means background extraction so that the background is completely ignored when choosing the k-means clusters.

I would also advice you to transform your images to one reference image (example at the end below). This will make your life easier when you want to compare sets of images and you have transformed all those images to the same reference (using meanshape that can get quit complicated).

Let me know if this code works for you

# make list with images
IDlist <- c('PA_1_WW_02', 'PA_1_WW_03', 'PA_1_WW_04')
prepath <- 'photos/'
extension <- '_D_calibrated.jpg'
imagelist <- makeList(IDlist, 'image', prepath, extension)

# load landmarks

landmarkList_tps_scale_FIJI <- makeList(IDlist, 'landmark', prepath = 'landmarks/', extension = '_D_land.txt')
landmarkList_tps_scale_TPS <- makeList(IDlist, 'landmark', format='tps', tpsFile= './landmarks/practice_etc.TPS')


##### 3. Landmark registration and K-means cluster TPS file WITH SCALE ####

# sampleRGB(imagelist[[1]], resampleFactor = 10)
RGB <- c(105,90,72)
rasterList_lanK_tps_FIJI <- patLanRGB(imagelist, landmarkList_tps_scale_FIJI, RGB, colOffset = 0.15, adjustCoords = TRUE, plot= 'stack', transformRef = 'meanshape',
                                           resampleFactor = 3, res = 150, cropOffset = c(10,10,10,10))

rasterList_lanK_tps_TPS <- patLanRGB(imagelist, landmarkList_tps_scale_TPS, RGB, colOffset = 0.15, adjustCoords = FALSE, plot= 'stack', transformRef = 'meanshape',
                                           resampleFactor = 3, res = 150, cropOffset = c(10,10,10,10))

rasterList_lanK_tps_K <- patLanK(imagelist, landmarkList_tps_scale_TPS, k=3, adjustCoords = FALSE, plot=T, resampleFactor = 3, removebgK = 220, res = 150, cropOffset = c(10,10,10,10))

# sum the colorpatterns
summedRaster_lanK_tps_FIJI <- sumRaster(rasterList_lanK_tps_FIJI, IDlist, type = 'RGB')
summedRaster_lanK_tps_TPS <- sumRaster(rasterList_lanK_tps_TPS, IDlist, type = 'RGB')
summedRaster_lanK_tps_K <- sumRaster(rasterList_lanK_tps_K, IDlist, type = 'k')


# cartoon
cartoon <- read.table('cartoon/PA_1_WW_02_cartoon.txt',h=F)

# plot
colfunc <- inferno(100)
plotHeat(summedRaster_lanK_tps_FIJI, IDlist, colpalette = colfunc, adjustCoords = TRUE, refShape = 'mean', landList = landmarkList_tps_scale_FIJI,
         imageList = imagelist, outline = cartoon, cartoonID = 'PA_1_WW_02', plotCartoon = TRUE, cartoonFill = 'black', cartoonOrder = 'under')

plotHeat(summedRaster_lanK_tps_TPS, IDlist, colpalette = colfunc, adjustCoords = FALSE, refShape = 'mean', landList = landmarkList_tps_scale_TPS,
         imageList = imagelist, outline = cartoon, cartoonID = 'PA_1_WW_02', plotCartoon = TRUE, cartoonFill = 'black', cartoonOrder = 'under', format = 'tps')

plotHeat(summedRaster_lanK_tps_K, IDlist, colpalette = colfunc, adjustCoords = FALSE, refShape = 'mean', landList = landmarkList_tps_scale_TPS,
         imageList = imagelist, outline = cartoon, cartoonID = 'PA_1_WW_02', plotCartoon = TRUE, cartoonFill = 'black', cartoonOrder = 'under', format = 'tps')



# alternative k-means analysis that allows to mask using the outline (although masking the background works better for you I think)

## I'm working here with a target image, because if you ever want to compare sets of images this will work better than meanshape
target <- landmarkList_tps_scale_FIJI[[1]]
imageList_aligned_FIJI <- alignLan(imagelist, landmarkList_tps_scale_FIJI, transformRef = target, adjustCoords = TRUE,
                              plotTransformed = T, resampleFactor = 3, maskOutline = cartoon, cartoonID = 'PA_1_WW_02')

target <- landmarkList_tps_scale_TPS[[1]]
imageList_aligned_TPS <- alignLan(imagelist, landmarkList_tps_scale_TPS, transformRef = target, adjustCoords = FALSE,
                              plotTransformed = T, resampleFactor = 3, maskOutline = cartoon, cartoonID = 'PA_1_WW_02', format = 'tps')


rasterList_lanK_tps_K2 <- patK(imageList_aligned_TPS, k=3, plot = T, maskToNA = 0)

summedRaster_lanK_tps_K2 <- sumRaster(rasterList_lanK_tps_K2, IDlist, type = 'k')

plotHeat(summedRaster_lanK_tps_K2, IDlist, colpalette = colfunc, adjustCoords = FALSE, refShape = target, landList = landmarkList_tps_scale_TPS,
         imageList = imagelist, outline = cartoon, cartoonID = 'PA_1_WW_02', plotCartoon = TRUE, cartoonFill = 'black', cartoonOrder = 'under', format = 'tps')

[StevenVB12]

Hi Eve,

I can't reproduce your error from the script that I gave you earlier. In your TotalList, do all rasters have the same resolution?

# make list with images
IDlist <- c('PA_1_WW_02', 'PA_1_WW_03', 'PA_1_WW_04')
prepath <- 'photos/'
extension <- '_D_calibrated.jpg'
imagelist <- makeList(IDlist, 'image', prepath, extension)

# load landmarks

landmarkList_tps_scale_TPS <- makeList(IDlist, 'landmark', format='tps', tpsFile= './landmarks/practice_etc.TPS')


target <- landmarkList_tps_scale_TPS[[1]]
cartoon <- read.table('cartoon/PA_1_WW_02_cartoon.txt',h=F)

RGB <- c(105,90,72)
rasterList_lanK_tps_TPS <- patLanRGB(imagelist, landmarkList_tps_scale_TPS, RGB, colOffset = 0.15, adjustCoords = FALSE, plot= 'stack', transformRef = target,
                                     resampleFactor = 3, res = 150, cropOffset = c(10,10,10,10))

summedRaster_lanK_tps_TPS <- sumRaster(rasterList_lanK_tps_TPS, IDlist, type = 'RGB')


colfunc <- inferno(100)

plotHeat(summedRaster_lanK_tps_TPS, IDlist, colpalette = colfunc, adjustCoords = FALSE, refShape = 'target', landList = landmarkList_tps_scale_TPS,
         imageList = imagelist, outline = cartoon, cartoonID = 'PA_1_WW_02', plotCartoon = TRUE, cartoonFill = 'black', cartoonOrder = 'under', format = 'tps')


TotalList <- rasterList_lanK_tps_TPS
popList <- list(IDlist)
symbolList <- c('19')
colList <- c('red')
pcaOut <- patPCA(TotalList, popList, colList, symbolList = symbolList, plot = TRUE, plotType = 'points', 
                 plotChanges = TRUE, PCx = 1, PCy = 2, plotCartoon = TRUE, refShape = 'target', outline = cartoon, 
                 flipOutline = 'y', imageList = imagelist, cartoonID = 'PA_1_WW_02', 
                 normalized = TRUE, cartoonFill = 'black', cartoonOrder = 'under', legendTitle = 'Predicted')

[EveTC]

Hi Steven,

This script works perfectly! Thank you so much.

Can I just confirm that when you use registration instead of landmarks (as I am wishing to compare the outputs) you do not need the alignment step to enable comparisons between populations? The registraion does this itself to the provided target image?

From my preliminary investigations it seems the target image can change the output of a PCA from the registration method quite substaintially? As such how do you suggest to selcet the target image and should it be part of one of the populations you are comparing? Is this the advantage of using landmarks over registration?

Cheers

[StevenVB12]

Hi Eve,

with registration, you do not need landmarks. The registration looks for sharp patterns of contrast in the images and uses that to align them. That also means that if there is some artifact in your background that is very consistent, the algorithm might pick that artifact to align. Also, the registration only uses a linear transform (size, shear and orientation), whereas the landmarks uses TPS, which allows to fit more complex shape changes. The latter would explain the less perfect alignment in the following example:

target <- imagelist[['PA_1_WW_02']]

RGB <- c(105,90,72)
rasterList_regRGB <- patRegRGB(imagelist, target, RGB, colOffset = 0.15, plot= 'stack', resampleFactor = 3)

summedRaster_regRGB <- sumRaster(rasterList_regRGB, IDlist, type = 'RGB')


colfunc <- inferno(100)

plotHeat(summedRaster_regRGB, IDlist, plotCartoon = TRUE, refShape = 'target', outline = cartoon, 
         imageList = imagelist, cartoonID = 'PA_1_WW_02', flipOutline = 'y', flipRaster = 'x',
         cartoonFill = 'black', cartoonOrder = 'under', colpalette = colfunc)

TotalList <- rasterList_regRGB
popList <- list(IDlist)
symbolList <- c('19')
colList <- c('red')
pcaOut <- patPCA(TotalList, popList, colList, symbolList = symbolList, plot = TRUE, plotType = 'points', 
                 plotChanges = TRUE, PCx = 1, PCy = 2, plotCartoon = TRUE, refShape = 'target', outline = cartoon, 
                 imageList = imagelist, cartoonID = 'PA_1_WW_02', flipOutline = 'y', flipRaster = 'x', 
                 normalized = TRUE, cartoonFill = 'black', cartoonOrder = 'under', legendTitle = 'Predicted')

[StevenVB12]

For registration, you should always use the same target image if you want to compare populations.

Hope this helps.

Steven

[EveTC]

Hi Steven,

Ok, thank you for your help. I realised that the error was because one of my populations had a different resample factor.

I have just tried to increase the number of my samples using landmarks following your example script. However when I run imageList_aligned_TPS <- alignLan(imageList_CoreNorth_LF, landmarkList_core_n_TPS, transformRef = target, adjustCoords = FALSE, plotTransformed = T, resampleFactor = 3, maskOutline = outline_LF_CNM6_D, cartoonID = 'CNM6_D', format = 'tps') and recieve this error: Error in raster::extent(imagelist[[cartoonID]]) : object 'imagelist' not found

Although I have set the imageList by this: imageList_CoreNorth_LF <- makeList(IDlist_CoreNorth, 'image', prepath_CNM_dorsal_crop, extension)

I have checked that the IDList samples are in the same order as those in the TPS file. ANy ideas where I made a mistake?

[StevenVB12]

Hi, my best guess is the the cartoonID 'CNM6_D' is not among the images in the imagelist.

Steven

[EveTC]

Its definetly there. Ill keep playing and let you know if I find the solution. Thank you for all your help Steven

Eve

[StevenVB12]

If you keep getting the error, please share me reproducible code on dropbox.

Cheers, Steven

[EveTC]

I can't work out where the error is coming from. I'm so sorry to be a pain!

I have shared a dropbox folder with you, using the email address you provided before.

[StevenVB12]

Hi Eve,

sorry if you have been spending a lot of time on this, because this was totally caused by typos in the added tps code! (please reinstall)

I do notice that your landmarks are scaled (in the range of 0-2). I'm afraid that won't work either as the XY coordinates should be be in pixel coordinates (in the range of 0-4000).

Cheers, Steven

[EveTC]

Hi Steven,

Ah Im am relieved as I thought it may have been a silly error that I made myself. No worries at all!

Oh I set them the scale in tpsDig. Does the code use the scale? Is this why the summedraster looks like this for me?

Also couple more questions (sorrry!):

1. Is there a way to use alignLan() with "remove background" instead of mask outline? You suggest that remove background is more suitable to me but I cant seem to use that and not the meanshape?
2. For Landmarks do I align populations to the same target individual for comparison like registration or seperately per population?

Thank you again Eve

[StevenVB12]

Hi Eve,

If you just want to mask the background, you should get the same result with patLanK as with alignLan + K.

I could add the meanshape option to alignLan if you want, but I think down the line things will be easier if you choose one image and use that as your reference. I would then align batches of samples that belong to the same population. The resulting rasterlists can then easily be compared or used in the PCA function and all match the same reference image. The latter is necessary to be able to compare populations.

The code does not use scale. But it shouldn't be to hard to get the coordinates from the scale: x = scale * raster::extent(image)[2] Y = scale * raster::extent(image)[4]

Steven

-------- Original message -------- From: EveTC [email protected] Date: 18/07/2019 04:38 (GMT-04:00) To: StevenVB12/patternize [email protected] Cc: "Steven M. Van Belleghem" [email protected], Comment [email protected] Subject: Re: [StevenVB12/patternize] Landmarks tps format (#15)

Hi Steven,

Ah Im am relieved as I thought it may have been a silly error that I made myself. No worries at all!

Oh I set them the scale in tpsDig. Does the code use the scale?

Also couple more questions (sorrry!):

1. Is there a way to use alignLan() with "remove background" instead of mask outline? You suggest that remove background is more suitable to me but I cant seem to use that and not the meanshape?
2. For Landmarks do I align populations to the same target individual like registration or seperately per population?

Thank you again Eve

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/StevenVB12/patternize/issues/15?email_source=notifications&email_token=ABQOC45PCJOO55IYCSI3Y3DQAATYZA5CNFSM4H7OVVJ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2HYKJQ#issuecomment-512722214, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABQOC4ZVTERDZXFK2LTGFWDQAATYZANCNFSM4H7OVVJQ.

[EveTC]

Fantastic, thank you so much. Yes don't worry about putting mean shape in thank you though. I think it's looking good!

Should the target individual be separate from all the populations, ie not included in the populations themselves?

[StevenVB12]

No specific requirement. I would take the one for which you drew the outline (has to be the same image).

Steven

-------- Original message -------- From: EveTC [email protected] Date: 18/07/2019 09:03 (GMT-04:00) To: StevenVB12/patternize [email protected] Cc: "Steven M. Van Belleghem" [email protected], Comment [email protected] Subject: Re: [StevenVB12/patternize] Landmarks tps format (#15)

Fantastic, thank you so much.

Should the target individual be separate from all the populations, ie not included in the populations themselves?

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/StevenVB12/patternize/issues/15?email_source=notifications&email_token=ABQOC47TXK7DL2LX67PPIV3QABS3BA5CNFSM4H7OVVJ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2IMY4I#issuecomment-512806001, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABQOC45I5D6LSFGOPMF2T6DQABS3BANCNFSM4H7OVVJQ.

[EveTC]

Hi Steven, Thank you so much for all your help.

In the alignLan function I recieve this error if the target individual is not included in the imageList of a population: Error in (function (classes, fdef, mtable) : unable to find an inherited method for function ‘extent’ for signature ‘"NULL"’ Does this mean I have to include the target individual in both populations? Surely this will mean that in the PCA output one point will be identical and skew the points towards similarity?

[StevenVB12]

Hi Eve,

Sorry, I hadn't considered this option. I think I've fixed this.

You can now load a reference image as:

targetIm <- raster::stack("photos/CNM1_D_calibrated.jpg")

And then set cartoonID = NULL and refImage = targetIm.

imageList_aligned_TPS_noscale <- alignLan(imagelist_CN[c(2:4)], landmarkList_CN_noScale_TPS[c(2:4)], transformRef = target, = FALSE, plotTransformed = T, resampleFactor = 10, maskOutline = cartoon,                                           cartoonID = NULL, format = 'tps', refImage = targetIm)

Let me know if it works!

Steven

[EveTC]

Hi Steven,

No worries, thank you for your speedy response again. Does this image need landmarks of its own?

Cheers, Eve

[StevenVB12]

Yes, you still have to provide the landmarks to transformRef.

Cheers, Steven

---

From: EveTC [email protected] Sent: Monday, July 22, 2019 2:27:02 PM To: StevenVB12/patternize [email protected] Cc: Steven M. Van Belleghem [email protected]; Comment [email protected] Subject: Re: [StevenVB12/patternize] Landmarks tps format (#15)

Hi Steven,

No worries, thank you for your speedy response again. Does this image need landmarks of its own?

Cheers, Eve

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/StevenVB12/patternize/issues/15?email_source=notifications&email_token=ABQOC4YIJKSYKXN5OY24QULQAX3XNA5CNFSM4H7OVVJ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2QXYOA#issuecomment-513899576, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABQOC454SY2LJP5BHDW2RELQAX3XNANCNFSM4H7OVVJQ.

[EveTC]

Hi Steven,

It is working perfectly up until plotHeat() where I get this error: Error in imageList[[cartoonID]] : attempt to select less than one element in get1index

Cheers, Eve