eclipse-platform
Project description links information is not reconciled on project close and reopen
In our product, we have a builder that reads linked folders. The builder runs into a NPE, due to stale links information. The stale links information results due to the following sequence:
- Debug Eclipse with a breakpoint at the start of
Project.reconcileLinksAndGroups(). - Create a Java project.
- Add a linked source folder to some folder on disk.
- Close the project.
- Delete the information about the link from the .project file on disk, e.g. delete the following block (location will be different):
<linkedResources>
<link>
<name>test</name>
<type>2</type>
<location>/data/tmp/test</location>
</link>
</linkedResources>
- Open the project in Eclipse.
- Observe
oldDescription.getLinks()returnsnulland so nothing is done to refresh the links information.
The information about links in the project description is not persisted in LocalMetaArea.writePrivateDescription() and is also not read in LocalMetaArea.readPrivateDescription(). We should fix this, to ensure up-to-date links information in the project description after opening a closed project.
5. Delete the information about the link from the .project file on disk, e.g. delete the following block
Just wondering, why should project still point to the link if this information is explicitly deleted from .project? Do I miss something?
Just wondering, why should project still point to the link if this information is explicitly deleted from .project? Do I miss something?
I don't think it should, but the link information is stale and not updated (this bug). Our product is using classpath information (the claspath for whatever reason remains the same despite the .project change), for a classpath entry the code asks the project if a folder is linked, the project replies with "yes" (this bug).
Seems like LocalMetaArea.readPrivateDescription() is for legacy state. It doesn't run any code now, it just exits early. Instead the description is read with this line:
description = new ProjectDescriptionReader(target).read(new InputSource(in));
I.e. it reads the current disk state.
The old description is supposed to be the one in memory. Unfortunately its no longer available at the time of the re-open.
After the re-open, a project description is created here:
"Worker-33: Open Project" #115 prio=5 os_prio=0 cpu=47.37ms elapsed=104.97s tid=0x00007ffe5c997f80 nid=0x10b48 at breakpoint [0x00007ffe9cef4000]
java.lang.Thread.State: RUNNABLE
at org.eclipse.core.internal.resources.ProjectDescription.<init>(ProjectDescription.java:107)
at org.eclipse.core.internal.resources.ProjectDescriptionReader.startElement(ProjectDescriptionReader.java:990)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement([email protected]/AbstractSAXParser.java:518)
at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement([email protected]/XMLNSDocumentScannerImpl.java:374)
at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl$NSContentDriver.scanRootElementHook([email protected]/XMLNSDocumentScannerImpl.java:613)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next([email protected]/XMLDocumentFragmentScannerImpl.java:3079)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDriver.next([email protected]/XMLDocumentScannerImpl.java:836)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next([email protected]/XMLDocumentScannerImpl.java:605)
at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next([email protected]/XMLNSDocumentScannerImpl.java:112)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument([email protected]/XMLDocumentFragmentScannerImpl.java:542)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse([email protected]/XML11Configuration.java:889)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse([email protected]/XML11Configuration.java:825)
at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse([email protected]/XMLParser.java:141)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse([email protected]/AbstractSAXParser.java:1224)
at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse([email protected]/SAXParserImpl.java:637)
at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.parse([email protected]/SAXParserImpl.java:326)
at org.eclipse.core.internal.resources.ProjectDescriptionReader.read(ProjectDescriptionReader.java:935)
at org.eclipse.core.internal.localstore.FileSystemResourceManager.read(FileSystemResourceManager.java:897)
at org.eclipse.core.internal.resources.SaveManager.restoreMetaInfo(SaveManager.java:894)
at org.eclipse.core.internal.resources.SaveManager.restore(SaveManager.java:769)
at org.eclipse.core.internal.resources.Project.open(Project.java:1067)
at org.eclipse.ui.actions.OpenResourceAction$1.doOpenWithReferences(OpenResourceAction.java:233)
at org.eclipse.ui.actions.OpenResourceAction$1.runInWorkspace(OpenResourceAction.java:279)
at org.eclipse.core.internal.resources.InternalWorkspaceJob.run(InternalWorkspaceJob.java:43)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)
Another description is then created with this call:
"Worker-31: Refreshing workspace" #111 prio=5 os_prio=0 cpu=91.29ms elapsed=867.35s tid=0x00007ffebc1aa000 nid=0x103df at breakpoint [0x00007ffe9f5f3000]
java.lang.Thread.State: RUNNABLE
at org.eclipse.core.internal.resources.ProjectDescription.<init>(ProjectDescription.java:107)
at org.eclipse.core.internal.resources.ProjectDescriptionReader.startElement(ProjectDescriptionReader.java:990)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement([email protected]/AbstractSAXParser.java:518)
at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement([email protected]/XMLNSDocumentScannerImpl.java:374)
at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl$NSContentDriver.scanRootElementHook([email protected]/XMLNSDocumentScannerImpl.java:613)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next([email protected]/XMLDocumentFragmentScannerImpl.java:3079)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDriver.next([email protected]/XMLDocumentScannerImpl.java:836)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next([email protected]/XMLDocumentScannerImpl.java:605)
at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next([email protected]/XMLNSDocumentScannerImpl.java:112)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument([email protected]/XMLDocumentFragmentScannerImpl.java:542)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse([email protected]/XML11Configuration.java:889)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse([email protected]/XML11Configuration.java:825)
at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse([email protected]/XMLParser.java:141)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse([email protected]/AbstractSAXParser.java:1224)
at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse([email protected]/SAXParserImpl.java:637)
at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.parse([email protected]/SAXParserImpl.java:326)
at org.eclipse.core.internal.resources.ProjectDescriptionReader.read(ProjectDescriptionReader.java:935)
at org.eclipse.core.internal.localstore.FileSystemResourceManager.read(FileSystemResourceManager.java:897)
at org.eclipse.core.internal.resources.Project.updateDescription(Project.java:1384)
at org.eclipse.core.internal.resources.File.updateMetadataFiles(File.java:379)
at org.eclipse.core.internal.localstore.RefreshLocalVisitor.visit(RefreshLocalVisitor.java:306)
at org.eclipse.core.internal.localstore.UnifiedTree.accept(UnifiedTree.java:119)
at org.eclipse.core.internal.localstore.FileSystemResourceManager.refreshResource(FileSystemResourceManager.java:978)
at org.eclipse.core.internal.localstore.FileSystemResourceManager.refresh(FileSystemResourceManager.java:961)
at org.eclipse.core.internal.resources.Resource.refreshLocal(Resource.java:1573)
at org.eclipse.core.internal.refresh.RefreshJob.runInWorkspace(RefreshJob.java:228)
at org.eclipse.core.internal.resources.InternalWorkspaceJob.run(InternalWorkspaceJob.java:43)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)
So the new description is the last one, created when opening the project. And the previous (old) description is also read from disk when opening the project (and so has no links information), as opposed to using the in-memory description (that had links information).
The in-memory description is gone after the close, e.g. if I add this code, there is no old description at the time of this new reconcile call:
diff --git a/resources/bundles/org.eclipse.core.resources/src/org/eclipse/core/internal/resources/SaveManager.java b/resources/bundles/org.eclipse.core.resources/src/org/eclipse/core/internal/resources/SaveManager.java
index 24b0df8412..794183bbf3 100644
--- a/resources/bundles/org.eclipse.core.resources/src/org/eclipse/core/internal/resources/SaveManager.java
+++ b/resources/bundles/org.eclipse.core.resources/src/org/eclipse/core/internal/resources/SaveManager.java
@@ -908,6 +908,9 @@ public class SaveManager implements IElementInfoFlattener, IManager, IStringPool
//try to read private metadata and add to the description
workspace.getMetaArea().readPrivateDescription(project, description);
}
+ if (project.isOpen()) {
+ project.reconcileLinksAndGroups(description);
+ }
project.internalSetDescription(description, false);
if (failure != null) {
try {
Considering there is no old state to compare to, I wonder if we want to delete the linked folder resources (the project children, not the actual target folders) from the workspace on project close. If they are re-created on project open (due to being read from .project) then this is maybe OK to do... I did try to clear the linked flag tom the resource info of the linked folders but that somehow didn't work. Will have to double check why it doesn't help in our product.
Then again there already is this code in Project.internalClose():
// remove each member from the resource tree.
// DO NOT use resource.delete() as this will delete it from disk as well.
IResource[] members = members(IContainer.INCLUDE_PHANTOMS | IContainer.INCLUDE_TEAM_PRIVATE_MEMBERS | IContainer.INCLUDE_HIDDEN);
subMonitor.setWorkRemaining(members.length);
for (IResource member2 : members) {
Resource member = (Resource) member2;
workspace.deleteResource(member);
subMonitor.worked(1);
}
I guess the next step here is to add a test, so that we are not depending on our product to reproduce. Making it easier to debug and understand what resource infos exist and why they are stale.
With the current state of affairs, deleting links from .project file by external file modification isn't supported and "leaks" any linked folder added before.
While adding "linked" folders one has provided a way to create them automatically (by reading .project file) but didn't provided a 100% safe way to delete them automatically - it works only if they were removed by Eclipse UI or by editing the file in Eclipse.
The use case not working is "external delete" by modifying .project file outside of Eclipse while project is closed or Eclipse is not open at all.
<?xml version="1.0" encoding="UTF-8"?>
<projectDescription>
<name>A</name>
<comment></comment>
<projects>
</projects>
<buildSpec>
<buildCommand>
<name>org.eclipse.jdt.core.javabuilder</name>
<arguments>
</arguments>
</buildCommand>
<buildCommand>
<name>edu.umd.cs.findbugs.plugin.eclipse.findbugsBuilder</name>
<arguments>
</arguments>
</buildCommand>
</buildSpec>
<natures>
<nature>org.eclipse.jdt.core.javanature</nature>
<nature>edu.umd.cs.findbugs.plugin.eclipse.findbugsNature</nature>
</natures>
<!-- this part has to be removed while project is closed or Eclipse not started -->
<linkedResources>
<link>
<name>wsp_428</name>
<type>2</type>
<location>/tmp/wsp_428</location>
</link>
</linkedResources>
</projectDescription>
The "externally deleted" linked folder has following implications:
- In Explorers, it is shown but with a warning overlay (
org.eclipse.ui.internal.ide.LinkedResourceDecorator.decorate()) - In Explorers, for linked folders, content (children) can't be seen.
org.eclipse.core.filesystem.IFileInfosays it does NOT existsorg.eclipse.core.resources.IResource.exists()says it exists
Looking at the code, I assume org.eclipse.core.internal.resources.Project.reconcileLinksAndGroups(ProjectDescription) should not only check content of the .project file but also consider to visit all the existing links and check them for corresponding entry in the .project file (== ProjectDescription.getLinks() match).
I believe proposed PR doesn't cover the use case we have completely, as it only works on open/close of projects. So it can't detect changes made while Eclipse was closed (e.g. close Eclipse, change .project file to remove links, open Eclipse).
What we need is to detect external changes independently on open/close/not started Eclipse.
I believe the change should try to hook into the project restore operation which happens both on startup and project open
SaveManager.restore(Project, IProgressMonitor) and
SaveManager.restoreMetaInfo(Project, IProgressMonitor).
There we could try to use knowledge we have from .project timestamp (change should indicate links reconcile is needed)
FileSystemResourceManager.read(IProject, boolean)FileSystemResourceManager.isDescriptionSynchronized(IProject)
To reconcile links info we can brute force traverse whole tree and check if the found links are known by ProjectDescription.getLinks().
However, the more elegant (and scalable) solution would be to store ProjectDescription.getLinks() info into the .metadata (new file?) and read it back on project restore, comparing old .metadata state vs found .project state. Later should win and all "extra" links existing in .metadata should be removed from the resource tree (via oldLinkResource.delete(IResource.NONE, null), similar how it is done in Project.reconcileLinksAndGroups(ProjectDescription)).