Undergraduate Researcher

The Center for Emerging Technology Identification for Roadway Infrastructure Construction & Maintenancehas developed a radar and rubric to evaluate emerging technologies for VTRC scientists to pilot and research.

Thisrubrichas undergone several iterations of refinementthroughdiscussions with facultyanditerations withLLMs.The existing version of the radar has been createdusing aretrievalaugmentedgeneration(RAG)concept. The LLMcompiles 30 URLs for each technology andusesthemto rate the technology against the rubric. These scores are then used to place the technology on the radar.Around 90 technologies have been rated; these wereidentifiedusinga VTRC Report on Emerging Technologies in 2004and research conducted by theCenter’spostdoctoral researchers. 

As the Center shifts toward generating pilot projects for VTRC scientists,postdoctoral researchers,and undergraduate students to work on,there is still a need to verify the LLM outputandto builda more extensive database for each technology. The goal of this document is to outline a plan for undergraduate researchers hired by the Center to conduct this literature review andgenerate thesescoreswith thesupervision of the postdoctoral researchers.

Outline:

Undergraduateresearchershired by the Center are expected to conduct the following tasks:

• Verifythe URLsprovided by the LLM. If these URLs are sufficient,noadditionalinformation is required.
• For URLs to be considered sufficient, they must meet the following criteria:Provide information about the performance of the technology

• Provide information about the cost of the technology
• Provide informationsuch that the technology can be compared with conventional practice
• Come fromcrediblesources(e.g.,.govor.edusites or official vendor sites)

If the URLs are not sufficient, the undergraduate researchers must conduct anadditionalliterature search using academic papers orthe websites ofindustryvendorsofferingthe technology.

• Generate a summary based on theliterature review. The summary should define the technologyand provide context forthe rubricratings. The summary shouldcontainreferences and be no longer than one page.

• Ratetechnologies based onthe literaturereview. The ratings should be self-supporting(justifiable from the summary alone). If ratings are based onjudgmentdue to alack of information, justification must be provided.

• Justification must also be provided onany rubricrowsthatwere omittedfrom scoring.

• Provide a confidenceflag for each technology (high/medium/low)to guideadditionalreview. If confidence is medium or low, postdoctoral researchers will conduct anadditionalliterature reviewbefore signing off on scoring.

One of the postdoctoral researchers will sign off on each technology before the score is committed to the radar. If the postdoctoral researcher disagrees with therating, thetwowill meetto comparethe reasoning behindeachscoreanddeterminewhetheradditionalinformationshouldbe added to the summary.

Deliverables:

Undergraduate researchers willsubmitfor each technology,aone-pagePDFsummary along with arubric sheet highlighting the chosen scores and justification.A template will be provided for undergraduate researchers to fill out to ensure consistency.Both fileswill be stored in a folder named after each technology.