TL;DR#
Designing solutions for complex engineering requirements is crucial, but existing retrieval-augmented generation (RAG) methods fall short. There is a lack of benchmarks to evaluate if systems can generate complete, feasible solutions with constraints. The Long-form QA or Multi-hop QA focused on assembling knowledge fragments, missing the demands of complete solutions. The goal of automatically generate reliable solutions for these complex requirements remains unmet and necessitates new approaches and benchmarks to measure progress in this area.
This paper introduces SolutionBench, a new benchmark, to tackle the challenge, evaluating systems in generating complete solutions for complex engineering requirements. The study also presents SolutionRAG, a novel system with tree-based exploration and bi-point thinking to generate reliable solutions. Experimental results on SolutionBench show SolutionRAG outperforms existing methods, achieving state-of-the-art performance. The study has opened a new direction for real-world applications.
Key Takeaways#
Why does it matter?#
This paper introduces SolutionBench, a new benchmark for engineering solution design, and SolutionRAG, a novel system using tree-based exploration and bi-point thinking. The work addresses a significant gap in RAG research, offering a valuable tool and method to enhance automation and reliability in complex engineering solution design. It opens avenues for exploring advanced RAG techniques and their application to real-world problems.
Visual Insights#
🔼 This figure illustrates the task of complex engineering solution design, which involves generating complete and feasible solutions under multiple real-world constraints. It introduces SolutionRAG, a novel system designed to tackle this challenge. SolutionRAG utilizes a ‘bi-point thinking tree’ approach, where the system iteratively designs solutions and incorporates feedback through review, refining the solution until it meets the specified requirements. The bi-point tree visually represents this iterative solution generation and refinement process.
read the caption
Figure 1: This paper proposes the complex engineering solution design task and a new system that can generate reliable solutions via the bi-point thinking tree.
| Engineering Domain | # Datapoint | # Knowledge |
| Environment (Env.) | 119 | 554 |
| Mining (Min.) | 117 | 543 |
| Transportation (Tra.) | 124 | 870 |
| Aerospace (Aer.) | 115 | 802 |
| Telecom (Tel.) | 116 | 840 |
| Architecture (Arc.) | 118 | 858 |
| Water Resource (Wat.) | 119 | 802 |
| Farming (Far.) | 122 | 868 |
🔼 SolutionBench is a benchmark dataset for evaluating complex engineering solution design. This table presents the statistics of SolutionBench, showing the number of data points (representing real-world engineering problems with solutions) and the number of knowledge entries (facts and insights related to these domains) for each of the eight engineering domains included in the benchmark: Environment, Mining, Transportation, Aerospace, Telecom, Architecture, Water Resource, and Farming.
read the caption
Table 1: Statistics of the SolutionBench, which include data and knowledge across eight engineering domains. The number of datapoints in dataset and the number of knowledge in knowledge base are shown above.
In-depth insights#
RAG vs Solution#
Retrieval-Augmented Generation (RAG) focuses on augmenting language models with external knowledge to improve the accuracy and relevance of generated content. In contrast, Solution-oriented approaches like the paper’s SolutionRAG aim to generate complete and feasible solutions to complex problems, especially in engineering. While RAG primarily assembles existing knowledge, SolutionRAG emphasizes reasoning, design, and problem-solving to meet specific constraints. Existing RAG methods have been found to not generate satisfactory solutions, whereas SolutionRAG proves to be a more advanced approach.
Bi-point thinking#
Bi-point thinking in the context of complex engineering solution design, as presented in this paper, likely refers to a dual-perspective approach that intertwines solution generation and evaluation. It seems like there is an iterative process involving the creation of a potential solution followed by a critical review or commentary on that solution. This two-pronged strategy aims to address the multifaceted challenges inherent in engineering tasks, which usually contains many constraints. By alternating between designing and evaluating, the system can refine solutions more effectively, ensuring they are both complete and feasible. This method allows for the incorporation of feedback and the identification of potential issues that might be overlooked in a single-pass design process. The use of the method helps improve the reliability of generated solutions.
SolutionBench#
The ‘SolutionBench’ section introduces a new benchmark for evaluating systems in designing solutions for complex engineering requirements. It addresses a gap in existing RAG research, which has not sufficiently explored tasks with multiple real-world constraints demanding complete and feasible solutions. The section highlights the process of constructing this benchmark, emphasizing the importance of authoritative data sources and domain diversity to ensure credibility and comprehensive evaluation. Technical reports are collected from engineering journals and processed through template-based extraction using LLMs, followed by manual verification and redundancy removal. This ensures the benchmark accurately reflects real-world scenarios and provides a valuable tool for assessing the capabilities of systems like SolutionRAG in automating complex engineering solution design.
Solution Tree#
The “solution tree” concept, though not explicitly stated as a heading in this paper, can be inferred from the SolutionRAG framework. It explores multiple potential solutions to a complex engineering design problem. Rather than adhering to a fixed reasoning path, the system branches out. Each branch is assessed, and unpromising paths are pruned. The core concept is to enhance solution reliability by considering diverse approaches. This systematic exploration enables the model to escape local optima and converge towards an optimized design. The framework uses bi-point thinking to refine solutions, indicating nodes are split into design and review, thus iteratively improving quality. Pruning allows for efficiency.
RAG enhanced#
The paper extensively explores Retrieval-Augmented Generation (RAG) to address the complexities of engineering solution design. A key focus appears to be on enhancing traditional RAG frameworks to overcome limitations when dealing with multifaceted, real-world constraints inherent in engineering problems. SolutionRAG, a novel system is designed to improve the solution iteratively through tree-based exploration and bi-point thinking, alternating between solution design and review to guarantee generated solutions satisfy all constraints. This contrasts with standard RAG approaches that may struggle to produce feasible and complete solutions given the intricate requirements and constraints. The research highlights the inadequacy of relying solely on internal knowledge within LLMs, indicating a need for RAG-based methods that can effectively integrate external knowledge to tackle engineering challenges. Furthermore, the system uses a pruning mechanism to balance efficiency and performance. Overall, the paper emphasizes advancing RAG techniques to automate and enhance the reliability of complex engineering solution design, presenting SolutionRAG as a significant step forward in the field.
More visual insights#
More on figures
🔼 SolutionBench is a benchmark dataset for evaluating complex engineering solution design. The figure details the process of its creation: First, technology reports from authoritative engineering journals are gathered to ensure quality. Second, a manually designed template is used with Large Language Models (LLMs) to extract crucial information from these reports. This information includes requirements, solutions, analysis, and technical details. Finally, the extracted information undergoes human verification to correct any errors or inconsistencies and merge data from the same engineering domain into a unified knowledge base, creating the SolutionBench.
read the caption
Figure 2: Illustration of the SolutionBench construction method, which includes collecting technology reports from engineering journals to ensure authority and authenticity, extracting useful content based on a manually formatted template and powerful LLMs, and finally harvesting the benchmark after manual verification and merging.
🔼 SolutionRAG uses a tree-based exploration strategy to iteratively refine solutions. Each node in the tree represents either a proposed solution or a reviewer comment on a solution. The process alternates between solution generation and review (bi-point thinking). This ensures solutions consider all constraints. A pruning mechanism removes less promising solution paths to improve efficiency and focuses on the most promising solutions.
read the caption
Figure 3: Illustration of SolutionRAG, we set the child number of each node as 2 for easy presentation above. SolutionRAG uses tree-based exploration to find optimal solution improvement process, bi-point thinking to guarantee generated solutions satisfy all constraints, and a pruning mechanism to balance efficiency and performance.
🔼 This figure visualizes the performance improvement of SolutionRAG over different layers of the tree-based exploration process. As the tree grows deeper (more inference steps are performed), the scores (both analytical and technical) of the generated solutions consistently increase. This demonstrates SolutionRAG’s capacity for iterative refinement and improved solution quality as the model explores more solution paths.
read the caption
Figure 4: Performance changes during the tree growth. The figure shows that scores become higher as the tree grows, proving SolutionRAG can indeed improve the solution scores as inference being deep.
🔼 This figure visualizes the effectiveness of the node evaluation mechanism used in the SolutionRAG system. The graph compares the scores of solution nodes that were retained during the pruning process versus those that were pruned. The results clearly show that retained nodes consistently have higher scores than pruned nodes, demonstrating that the node evaluation method successfully identifies and retains the most promising solution paths, improving efficiency and solution quality.
read the caption
Figure 5: Effectiveness of node evaluation mechanism. The figure shows that scores in retained nodes are higher than in pruned nodes, thus the node evaluation is an effective method for judging and pruning in SolutionRAG.
🔼 This figure displays the template used to extract relevant information from engineering reports for a benchmark dataset. The template is designed to capture key aspects of the engineering design process, including real-world problem requirements, expert solutions, the analytical reasoning behind those solutions, the technical knowledge utilized, and the step-by-step explanation of the design process. This structured approach ensures consistency and completeness in the collected data.
read the caption
Figure 6: Template used to extract useful content from original engineering reports, aiming to capture real-world complex requirements, expert-authored solutions, analytical knowledge used to interpret the requirements, technical knowledge applied in addressing the requirements, and explanations for the expert’s solution design process.
🔼 Figure 7 details the prompts used in SolutionRAG’s tree-based exploration process. It shows how SolutionRAG generates new solution and comment nodes at each step. Starting from the root node (the problem requirement), prompts guide the system to generate solution proposals. Subsequently, prompts are used when evaluating these solutions to generate comments highlighting areas for improvement. Further prompts drive the iterative refinement of solutions based on the comments. The process repeats, alternating between solution proposals and comments to gradually build a reliable and complete solution.
read the caption
Figure 7: Prompts used in node expansion of tree growth, including generating solution proposals and solutions based on the root node, generating comment proposals and comments based on a solution node, and generating solution proposals and solutions based on a comment node.
🔼 Figure 8 presents the prompts used to evaluate the quality of a system’s solution to a complex engineering problem. These prompts leverage GPT-4 to assess two key aspects: (1) the analytical score, which evaluates the system’s understanding and consideration of the complex constraints, and (2) the technical score, which assesses the appropriateness and accuracy of the technologies applied. The evaluation process uses the ‘gold standard’ solution, explanation, analytical knowledge, and technical knowledge as references, allowing for a comprehensive comparison and a numerical score (0-100) for each aspect.
read the caption
Figure 8: Prompts for calculating analytical score and technical score, which uses the golden solution, explanation, and corresponding analytical and technical knowledge as references, allowing GPT-4o to assess whether the system’s solution sufficiently consider the challenges posed by the complex constraints and apply the appropriate technologies to address the complex constraints in the requirements.
More on tables
| Method | Env. | Min. | Tra. | Aer. | Tel. | Arc. | Wat. | Far. | ||||||||
| AS | TS | AS | TS | AS | TS | AS | TS | AS | TS | AS | TS | AS | TS | AS | TS | |
| Deep Reasoning Models | ||||||||||||||||
| o1-2024-12-17 OpenAI (2024b) | 60.5 | 48.3 | 51.9 | 37.5 | 57.3 | 44.7 | 57.8 | 47.6 | 63.5 | 52.3 | 61.2 | 52.0 | 59.9 | 50.4 | 62.9 | 52.2 |
| GLM-Zero-Preview Zhipu (2024) | 47.0 | 30.6 | 43.2 | 22.2 | 45.2 | 27.0 | 42.3 | 25.7 | 45.1 | 31.7 | 47.7 | 32.4 | 47.3 | 30.8 | 51.4 | 36.6 |
| QwQ-32B-Preview Qwen (2024) | 54.3 | 38.7 | 48.0 | 27.9 | 47.2 | 29.3 | 47.4 | 31.9 | 52.2 | 35.9 | 51.3 | 35.6 | 49.2 | 33.0 | 53.4 | 37.0 |
| Single-round RAG Methods | ||||||||||||||||
| Naïve-RAG Lewis et al. (2020) | 64.8 | 62.2 | 57.2 | 40.1 | 62.7 | 54.9 | 67.7 | 65.4 | 67.4 | 66.8 | 66.2 | 63.3 | 66.0 | 57.5 | 65.7 | 63.0 |
| Rerank-RAG Li et al. (2023) | 62.7 | 60.7 | 53.4 | 38.4 | 60.0 | 49.7 | 65.6 | 65.2 | 66.1 | 63.4 | 66.4 | 62.8 | 64.1 | 55.4 | 64.0 | 59.7 |
| Multi-round RAG Methods | ||||||||||||||||
| Self-RAG Asai et al. (2024) | 64.2 | 63.6 | 56.1 | 41.6 | 62.9 | 56.5 | 68.8 | 69.9 | 67.6 | 66.9 | 66.7 | 65.9 | 64.8 | 58.6 | 65.1 | 61.1 |
| GenGround Shi et al. (2024) | 54.8 | 46.1 | 53.0 | 33.3 | 54.7 | 37.2 | 55.7 | 46.0 | 58.3 | 50.7 | 60.1 | 50.7 | 60.4 | 48.9 | 59.8 | 52.7 |
| RQ-RAG Chan et al. (2024) | 53.5 | 44.4 | 48.9 | 28.7 | 53.8 | 38.8 | 55.0 | 46.1 | 57.9 | 44.6 | 56.3 | 46.9 | 54.3 | 39.8 | 57.2 | 45.2 |
| Tree-based Exploration and Bi-point Thinking | ||||||||||||||||
| SolutionRAG (Ours) | 66.4 | 67.9 | 59.7 | 50.5 | 64.1 | 58.5 | 69.9 | 72.7 | 68.8 | 69.0 | 67.9 | 68.0 | 66.0 | 60.7 | 66.9 | 65.2 |
🔼 This table presents the main experimental results of evaluating various methods on SolutionBench, a benchmark dataset for complex engineering solution design. The benchmark includes eight different engineering domains. For each method and domain, two scores are reported: Analytical Score (AS) and Technical Score (TS), reflecting the system’s ability to produce solutions that are both analytically sound and technically feasible, respectively. The results highlight the significant performance gap between existing methods (including those based on deep reasoning and retrieval-augmented generation) and the proposed SolutionRAG system. SolutionRAG shows a substantial improvement in generating complete and reliable solutions for complex engineering design problems.
read the caption
Table 2: Main experimental results on SolutionBench with eight engineering domains, the AS is the analytical score and TS is the technical score. The table shows that previous methods perform poorly for complex engineering solution design. In contrast, our SolutionRAG is able to output more complete and reliable solutions.
| Method | Env. | Min. | Tra. | Aer. | Tel. | Arc. | Wat. | Far. | Overall | |||||||||
| AS | TS | AS | TS | AS | TS | AS | TS | AS | TS | AS | TS | AS | TS | AS | TS | AS | TS | |
| SolutionRAG | 66.4 | 67.9 | 59.7 | 50.5 | 64.1 | 58.5 | 69.9 | 72.7 | 68.8 | 69.0 | 67.9 | 68.0 | 66.0 | 60.7 | 66.9 | 65.2 | 66.2 | 64.1 |
| w/o tree structure | 63.5 | 66.5 | 57.3 | 46.2 | 63.1 | 57.4 | 60.8 | 68.4 | 60.9 | 63.7 | 66.2 | 67.2 | 65.6 | 59.9 | 64.2 | 63.9 | 62.7 | 61.7 |
| w/o bi-point thinking | 62.8 | 64.7 | 55.6 | 47.3 | 61.5 | 55.7 | 63.2 | 68.3 | 62.6 | 64.8 | 67.5 | 67.3 | 64.4 | 59.1 | 65.2 | 64.7 | 62.9 | 61.5 |
🔼 This ablation study investigates the individual contributions of the tree-based exploration and bi-point thinking mechanisms within the SolutionRAG system. The results demonstrate that both mechanisms significantly improve the overall performance of SolutionRAG in generating solutions for complex engineering problems. Notably, the ablation study indicates that both mechanisms have approximately equal importance to the system’s success, highlighting their synergistic effects.
read the caption
Table 3: Ablation results for tree-based exploration and bi-point thinking. The table shows that both mechanisms have obviously positive effects for SolutionRAG and exhibit a similar level of importance in the overall.
| Environment | |
| Journal Name | ISSN |
| Journal of Environmental Engineering Technology | 1674-991X |
| Environmental Sanitation Engineering | 1005-8206 |
| The Administration and Technique of Environmental Monitoring | 1006-2009 |
| Environment and Development | 2095-672X |
| Environmental Protection and Technology | 1674-0254 |
| Green Environmental Protection Building Materials | 1673-6680 |
| Journal of Henan University of Urban Construction | 1674-7046 |
| Urban Management and Science & Technology | 1008-2271 |
| Science and Technology Square | 1671-4792 |
| Construction Materials & Decoration | 1673-0038 |
| Intelligent City | 2096-1936 |
| Instrument Standardization & Metrology | 1672-5611 |
| Northwest Hydropower | 1006-2610 |
| Technology & Economics in Petrochemicals | 1674-1099 |
| Water Purification Technology | 1009-0177 |
| Construction Science and Technology | 1671-3915 |
| Urban Geology | 2097-3764 |
| Engineering and Construction | 1673-5781 |
| Engineering and Technological Research | 2096-2789 |
| Scientific and Technological Innovation | 2096-4390 |
| Engineering & Test | 1674-3407 |
| Inner Mongolia Water Resources | 1009-0088 |
| China Cement | 1671-8321 |
| Guangdong Chemical Industry | 1007-1865 |
| Jiangxi Building Materials | 1006-2890 |
| Tianjin Science & Technology | 1006-8945 |
| Journal of Zhejiang University of Water Resources and Electric Power | 2095-7092 |
| China Municipal Engineering | 1004-4655 |
| China Storage & Transport | 1005-0434 |
| Mining | |
| Journal Name | ISSN |
| Coal Engineering | 1671-0959 |
| Mining Engineering | 1671-8550 |
| Mechanical Management and Development | 1003-773X |
| Coal and Chemical Industry | 2095-5979 |
| Colliery Mechanical & Electrical Technology | 1001-0874 |
| Modern Mining | 1674-6082 |
| China Mine Engineering | 1672-609X |
| Shandong Coal Science and Technology | 1005-2801 |
| Jiangxi Coal Science & Technology | 1006-2572 |
| Metal Mine | 1001-1250 |
| Modern Chemical Research | 1672-8114 |
| Petroleum Geology and Engineering | 1673-8217 |
| Coal Mine Modernization | 1009-0797 |
| Shaanxi Coal | 1671-749X |
| Drilling Engineering | 2096-9686 |
| Mineral Resources and Geology | 1001-5663 |
| Mine Surveying | 1001-358X |
| Coal | 1005-2798 |
| Mining Equipment | 2095-1418 |
| Inner Mongolia Coal Economy | 1008-0155 |
| Inner Mongolia Petrochemical Industry | 1006-7981 |
| Energy and Energy Conservation | 2095-0802 |
| China Plant Engineering | 1671-0711 |
| Engineering and Construction | 1673-5781 |
| Scientific and Technological Innovation | 2096-4390 |
| Engineering & Test | 1674-3407 |
| Energy Technology and Management | 1672-9943 |
| Coal Technology | 1008-8725 |
🔼 This table lists the engineering journals used to compile the SolutionBench benchmark dataset. It’s divided into sections, with environment and mining journals listed in Table 4, while transportation, aerospace, telecom, architecture, water resources, and farming journals are detailed in Tables 5 and 6 respectively. The inclusion of these diverse journals ensures a wide range of engineering domains are represented in the benchmark, providing a robust and comprehensive evaluation of solution design systems.
read the caption
Table 4: List of the engineering journals used for construction the benchmark. The information for environment domain and mining domain is shown above, and information for other domains is in Table 5 and 6.
| Transportation | |
| Journal Name | ISSN |
| Railway Construction Technology | 1009-4539 |
| Northern Communications | 1673-6052 |
| China Municipal Engineering | 1004-4655 |
| Highway | 0451-0712 |
| Urban Roads Bridges & Flood Control | 1009-7716 |
| Technology Innovation and Application | 2095-2945 |
| Marine Equipment/Materials & Marketing | 1006-6969 |
| Engineering and Construction | 1673-5781 |
| Port Operation | 1000-8969 |
| Structural Engineers | 1005-0159 |
| China Highway | 1006-3897 |
| Engineering and Technological Research | 2096-2789 |
| Construction Machinery Technology & Management | 1004-0005 |
| TranspoWorld | 1006-8872 |
| Railway Investigation and Surveying | 1672-7479 |
| Transport Construction & Management | 1673-8098 |
| Guangdong Water Resources and Hydropower | 1008-0112 |
| Western China Communications Science & Technology | 1673-4874 |
| Jiangsu Science and Technology Information | 1004-7530 |
| Value Engineering | 1006-4311 |
| Hoisting and Conveying Machinery | 1001-0785 |
| Jiangxi Building Materials | 1006-2890 |
| Scientific and Technological Innovation | 2096-4390 |
| Transport Business China | 1673-3681 |
| Sichuan Cement | 0451-0712 |
| Aerospace | |
| Journal Name | ISSN |
| Spacecraft Engineering | 1673-8748 |
| Aeronautical Manufacturing Technology | 1671-833X |
| Aviation Maintenance & Engineering | 1672-0989 |
| Journal of Ordnance Equipment Engineering | 2096-2304 |
| Aeroengine | 2096-2304 |
| Space International | 2096-2304 |
| Avionics Technology | 1006-141X |
| System Simulation Technology | 1673-1964 |
| Journal of Civil Aviation | 2096-4994 |
| Safety & EMC | 1005-9776 |
| Internal Combustion Engine & Parts | 1674-957X |
| Aeronautical Computing Technique | 1671-654X |
| Meteorological Science and Technology | 1671-6345 |
| Journal of Astronautics | 1000-1328 |
| Communications Technology | 1002-0802 |
| Laser & Optoelectronics Progress | 1006-4125 |
| Engineering & Test | 1674-3407 |
| Chinese Space Science and Technology | 1000-758X |
| Ship Electronic Engineering | 1672-9730 |
| China Science and Technology Information | 1672-9730 |
| Journal of Deep Space Exploration | 2096-9287 |
| China Educational Technology & Equipment | 1671-489X |
| Micromotors | 1671-489X |
| Spacecraft Recovery & Remote Sensing | 1009-8518 |
| Journal of Chengdu Aeronautic Polytechnic | 1671-4024 |
| Telecom | |
| Journal Name | ISSN |
| Systems Engineering and Electronics | 1001-506X |
| Electronic Technology & Software Engineering | 2095-5650 |
| Video Engineering | 1002-8692 |
| Telecom Engineering Technics and Standardization | 1008-5599 |
| Radio & Television Network | 2096-806X |
| Study on Optical Communications | 1005-8788 |
| Electronics Quality | 1003-0107 |
| Radio & Television Information | 1007-1997 |
| Changjiang Information & Communications | 2096-9759 |
| Automation in Petro-Chemical Industry | 1007-7324 |
| Telecommunications Science | 1000-0801 |
| Computer Knowledge and Technology | 1009-3044 |
| Journal of Electronics & Information Technology | 1009-5896 |
| Laser & Optoelectronics Progress | 1006-4125 |
| China Digital Cable TV | 1007-7022 |
| Radio Engineering | 1003-3106 |
| Journal of Beijing Electronic Science and Technology Institute | 1672-464X |
| Laser Journal | 0253-2743 |
| Designing Techniques of Posts and Telecommunications | 1007-3043 |
| Wireless Internet Science and Technology | 1672-6944 |
| Journal of University of South China(Science and Technology) | 1673-0062 |
| Audio Engineering | 1002-8684 |
| Automation Application | 1674-778X |
| Chinese Journal of Lasers | 0258-7025 |
| Journal of Smart Agriculture | 2096-9902 |
🔼 This table lists the engineering journals used to gather data for creating the SolutionBench benchmark. The journals represent a diverse range of engineering domains, ensuring a comprehensive and varied dataset for evaluating complex engineering solution design systems.
read the caption
Table 5: List of the engineering journals used for construction the benchmark.
| Architecture | |
| Journal Name | ISSN |
| Building Technology Development | 1001-523X |
| Building Structure | 1002-848X |
| Construction & Design for Engineering | 1007-9467 |
| Modern Paint & Finishing | 1007-9548 |
| Architecture Technology | 1000-4726 |
| Theoretical Research in Urban Construction | 2095-2104 |
| Urban Architecture Space | 2097-1141 |
| Art and Design | 1008-2832 |
| Architecture & Culture | 1672-4909 |
| Journal of Yangzhou Polytechnic College | 1008-3693 |
| Heating Ventilating & Air Conditioning | 1002-8501 |
| Construction Machinery & Maintenance | 1006-2114 |
| China Science and Technology Information | 1001-8972 |
| Construction Machinery and Equipment | 1000-1212 |
| Journal of Municipal Technology | 1009-7767 |
| Jiangxi Building Materials | 1006-2890 |
| Urban Roads Bridges & Flood Control | 1009-7716 |
| Fujian Construction Science & Technology | 1006-3943 |
| Sichuan Cement | 1007-6344 |
| Engineering and Technological Research | 2096-2789 |
| Journal of North China Institute of Science and Technology | 1672-7169 |
| Tianjin Construction Science and Technology | 1008-3197 |
| World Forestry Research | 1001-4241 |
| Jiangsu Building Materials | 1004-5538 |
| Shanghai Construction Science & Technology | 1005-6637 |
| Water Resource | |
| Journal Name | ISSN |
| Design of Water Resources & Hydroelectric Engineering | 1007-6980 |
| Hydro Science and Cold Zone Engineering | 2096-5419 |
| Journal of Water Resources and Architectural Engineering | 1672-1144 |
| Mechanical & Electrical Technique of Hydropower Station | 1672-5387 |
| Yangtze River | 1001-4179 |
| Port & Waterway Engineering | 1002-4972 |
| Technical Supervision in Water Resources | 1008-1305 |
| Small Hydro Power | 1007-7642 |
| Pearl River | 1001-9235 |
| Water Conservancy Construction and Management | 2097-0528 |
| Water Conservancy Science and Technology and Economy | 1006-7175 |
| Water Resources Planning and Design | 1672-2469 |
| Construction Quality | 1671-3702 |
| Henan Water Resources and South-to-North Water Diversion | 1673-8853 |
| Engineering and Construction | 1673-5781 |
| Technology and Market | 1006-8554 |
| Beijing Water | 1673-4637 |
| Port Engineering Technology | 2097-3519 |
| Water Resources & Hydropower of Northeast China | 1002-0624 |
| Mechanical and Electrical Information | 1671-0797 |
| Maritime Safety | 2097-1745 |
| Gansu Water Resources and Hydropower Technology | 2095-0144 |
| Water Power | 0559-9342 |
| Shanxi Water Resources | 1004-7042 |
| Haihe Water Resources | 1004-7328 |
| Farming | |
| Journal Name | ISSN |
| Modern Agricultural Science and Technology | 1007-5739 |
| Farm Machinery | 1000-9868 |
| Cereal & Feed Industry | 1003-6202 |
| Journal of Agricultural Mechanization Research | 1003-188X |
| Forestry Machinery & Woodworking Equipment | 2095-2953 |
| Transactions of the Chinese Society of Agricultural Engineering | 1002-6819 |
| Forest Research | 1001-1498 |
| Times Agricultural Machinery | 2095-980X |
| Protection Forest Science and Technology | 1005-5215 |
| Journal of Beijing University of Agriculture | 1002-3186 |
| Contemporary Horticulture | 1006-4958 |
| China Southern Agricultural Machinery | 1672-3872 |
| Forest Inventory and Planning | 1671-3168 |
| Agricultural Machinery Using & Maintenance | 2097-4515 |
| Journal of Green Science and Technology | 1674-9944 |
| China Forest Products Industry | 1001-5299 |
| Forestry Machinery & Woodworking Equipment | 2095-2953 |
| The Food Industry | 1004-471X |
| Journal of Hebei Forestry Science and Technology | 1002-3356 |
| Electrical Automation | 1000-3886 |
| Journal of Library and Information Science | 2096-1162 |
| Forest Science and Technology | 2097-0285 |
| Chinese Journal of Ecology | 1000-4890 |
| Popular Standardization | 1007-1350 |
| Management & Technology of SME | 1673-1069 |
🔼 This table lists the engineering journals used to gather data for creating the SolutionBench benchmark. The journals represent a diverse range of engineering domains, ensuring the benchmark data is comprehensive and representative of real-world scenarios.
read the caption
Table 6: List of the engineering journals used for construction the benchmark.
Full paper#
