Option to import xlsx should be available

[camilajenny]

1. Not being logged into the system and DCF account priviliges shouldn't restrict the ability to import a local catalogue.
2. An option to import xlsx directly should be possible
3. ECF format is artificial, it's being converted to XML and XLSX under the hood
4. XLSX is not open source, an option to import open formats like ODF should be supported
5. The whole solution is over-engineered

Devin says:

Summary

ECF is EFSA's proprietary catalogue format that contains XML data in a zip file. The application supports both ECF and XLSX formats, but they serve different purposes. The Tools menu is grayed out until you open a catalogue and have proper user permissions.

ECF Format

ECF (EFSA Catalogue Format) is a zip archive containing XML catalogue data. When importing an ECF file, the application:

1. Extracts the ECF file to get the inner XML file 1
2. Converts the XML to XLSX format internally 2
3. Imports the XLSX data into the database 3

The three supported import formats are defined in the ImportFileFormat enum: ECF, XML, XLSX 4 .

XLSX Support

XLSX files are indeed supported for import, but primarily for local catalogue editing. The "Import excel" function in the Tools menu specifically uses XLSX format 5 . This is considered an "advanced user" feature as mentioned in the manual because it's not supervised by EFSA.

Tools Menu Grayed Out

The Tools menu is disabled by default and only enabled when specific conditions are met:

1. Catalogue must be open: The menu is only enabled if mainMenu.getCatalogue() != null
2. User permissions checked: The user must not be in the process of getting user level (!user.isGettingUserLevel())
3. Individual item permissions: Each menu item has additional enablement logic based on user role (Catalogue Manager vs Data Provider) and catalogue state 6

To enable the Tools menu:

• Open or create a catalogue first (File menu)
• Log in with DCF credentials if you need catalogue management features
• For local catalogues, most Tools menu items will be available without login

Notes

• ECF is the standard format for distributing official EFSA catalogues
• XLSX is used for local imports/exports and manual editing
• The Tools menu grayed out is normal behavior until a catalogue is loaded
• Different menu items appear based on catalogue type (e.g., ICT options only for MTX catalogues) 7

Wiki pages you might want to explore:

• Menu System (openefsa/catalogue-browser)
• ICT (Interpreting and Checking Tool) (openefsa/catalogue-browser)

Citations

File: src/main/java/import_catalogue/CatalogueImporter.java (L61-63)

	public enum ImportFileFormat {
		ECF, XML, XLSX;
	}

File: src/main/java/import_catalogue/CatalogueImporter.java (L141-177)

	private String processEcf(String filename) {

		String outputFile = null;

		try {

			File inputFile = new File(filename);

			String outputFolder = GlobalUtil.getTempDir() + inputFile.getName() + "_unzip";

			// unzip the ecf file into the xml
			ZipManager.extractFolder(filename, outputFolder);

			final File unzippedFolder = new File(outputFolder);

			if (unzippedFolder.listFiles().length <= 0) {
				LOGGER.error("Wrong file format, " + "cannot find the xml file inside the .ecf");
				return null;
			}

			// add the unzipped folder to the garbage to
			// delete it at the end of the process
			garbage.add(unzippedFolder.getAbsolutePath());

			// get the xml file from the folder
			File xmlFile = unzippedFolder.listFiles()[0];

			outputFile = xmlFile.getAbsolutePath();

			return outputFile;

		} catch (IOException e1) {
			e1.printStackTrace();
		}

		return null;
	}

File: src/main/java/import_catalogue/CatalogueImporter.java (L186-197)

	private String processXml(String filename) throws TransformerException {

		String outputFilename = filename + ".xlsx";

		// convert the xml into an excel
		XmlCatalogueToExcel converter = new XmlCatalogueToExcel(filename, outputFilename);

		// do the conversion
		converter.convertXmlToExcel();

		return outputFilename;
	}

File: src/main/java/import_catalogue/CatalogueImporter.java (L235-247)

	 */
	private void importXml(String filename) throws TransformerException, IOException, XMLStreamException,
			OpenXML4JException, SAXException, SQLException, ImportException {

		String xlsxFile = processXml(filename);

		// at the end of the process delete the
		// temporary xlsx file
		garbage.add(xlsxFile);

		// import the xlsx catalogue
		importXlsx(xlsxFile);
	}

File: src/main/java/ui_main_menu/ToolsMenu.java (L166-176)

		// export for ICT (just for the MTX cat)
		if (mainMenu.getCatalogue() != null && mainMenu.getCatalogue().isMTXCatalogue()) {

			// check if ict file is present
			if (GlobalUtil.isIctInstalled()) {
				launchIctMI = addLaunchIct(toolsMenu);
				reInstallIctMI = addReInstallIct(toolsMenu);
			} else {
				installIctMI = addInstallIct(toolsMenu);
			}
		}

File: src/main/java/ui_main_menu/ToolsMenu.java (L500-526)

	private MenuItem addImportMI(Menu menu) {

		final MenuItem importItem = new MenuItem(menu, SWT.NONE);
		importItem.setText(CBMessages.getString("BrowserMenu.ImportCmd"));

		importItem.addSelectionListener(new SelectionAdapter() {
			@Override
			public void widgetSelected(SelectionEvent event) {

				final String filename = GlobalUtil.showExcelFileDialog(shell,
						CBMessages.getString("Import.ImportWindowTitle"), SWT.OPEN);

				// return if no filename retrieved
				if (filename == null || filename.isEmpty())
					return;

				// ask for final confirmation
				int val = GlobalUtil.showDialog(shell, CBMessages.getString("Import.ImportWarningTitle"),
						CBMessages.getString("Import.ImportWarningMessage"), SWT.OK | SWT.CANCEL | SWT.ICON_QUESTION);

				// return if cancel was pressed
				if (val == SWT.CANCEL)
					return;

				CatalogueImporterThread importCat = new CatalogueImporterThread(filename, ImportFileFormat.XLSX);

				importCat.setProgressBar(new FormProgressBar(shell, CBMessages.getString("Import.ImportXlsxBarTitle")));

File: src/main/java/ui_main_menu/ToolsMenu.java (L1239-1337)

		if (user.isCatManagerOf(mainMenu.getCatalogue())) {

			boolean isReservedByCurrentUser = user.hasReserved(mainMenu.getCatalogue().getCode());

			// if we are requesting a web service
			// disable the action which can send another
			// web service request
			if (user.hasPendingRequestsFor(mainMenu.getCatalogue())) {

				if (reserveMI != null) {
					reserveMI.setText(CBMessages.getString("Reserve.WaitingResponse"));
					reserveMI.setEnabled(false);
				}

				if (unreserveMI != null) {
					unreserveMI.setText(CBMessages.getString("Reserve.WaitingResponse"));
					unreserveMI.setEnabled(false);
				}

				if (publishMI != null) {
					publishMI.setText(CBMessages.getString("Reserve.WaitingResponse"));
					publishMI.setEnabled(false);
				}

				if (createXmlMI != null) {
					createXmlMI.setText(CBMessages.getString("Reserve.WaitingResponse"));
					createXmlMI.setEnabled(false);
				}

				if (uploadDataMI != null) {
					uploadDataMI.setText(CBMessages.getString("Reserve.WaitingResponse"));
					uploadDataMI.setEnabled(false);
				}

				// if we are reserving but we have forced the
				// editing, we leave these buttons enabled
				if (mainMenu.getCatalogue() != null && user.hasForcedReserveOf(mainMenu.getCatalogue()) == null) {

					if (resetMI != null)
						resetMI.setEnabled(false);

					if (importMI != null)
						importMI.setEnabled(false);

					if (hierarchyEditMI != null)
						hierarchyEditMI.setEnabled(false);

					if (attributeEditMI != null)
						attributeEditMI.setEnabled(false);
				}
			} else {

				if (reserveMI != null) {
					// can reserve only if not local and catalogue not reserved
					reserveMI.setText(CBMessages.getString("BrowserMenu.Reserve"));
					reserveMI.setEnabled(!isReservedByCurrentUser);
				}

				if (unreserveMI != null) {
					unreserveMI.setText(CBMessages.getString("BrowserMenu.Unreserve"));
					unreserveMI.setEnabled(isReservedByCurrentUser);
				}

				if (uploadDataMI != null) {
					uploadDataMI.setText(CBMessages.getString("BrowserMenu.UploadData"));

					// check if we have already created an xml file for this catalogue
					XmlUpdateFileDAO xmlDao = new XmlUpdateFileDAO();
					XmlUpdateFile xml = xmlDao.getById(mainMenu.getCatalogue().getId());
					uploadDataMI.setEnabled(
							mainMenu.getCatalogue().isLastRelease() && isReservedByCurrentUser && xml != null);
				}

				if (createXmlMI != null) {
					createXmlMI.setText(CBMessages.getString("BrowserMenu.CreateXml"));
					createXmlMI.setEnabled(mainMenu.getCatalogue().isLastRelease() && isReservedByCurrentUser);
				}

				if (publishMI != null) {
					publishMI.setText(CBMessages.getString("BrowserMenu.Publish"));
					publishMI.setEnabled(!isReservedByCurrentUser);
				}

				// if the catalogue can be modified
				if (canEdit) {

					// check if catalogue is reserved by the user and if it is an internal version
					boolean isReserved = mainMenu.getCatalogue().isReservedBy(user)
							&& mainMenu.getCatalogue().getCatalogueVersion().isInternalVersion();

					// enable resetMI
					if (resetMI != null) {
						resetMI.setEnabled(isReserved);
					}
				}

				if (importMI != null)
					importMI.setEnabled(canEdit);
			}