Grantee	Tsinghua University
Project Title	Developing a Collaborative BGP Routing Analyzing and Diagnosing Platform
Amount Awarded	USD 150,000
Dates covered by this report:	2022-01-28 to 2023-07-28
Economies where project was implemented	China
Project leader name	Jilong Wang
Project Team	Changqing An (China) Jie An (China) Hui Wang (China) Weiqi Zhao (China) Linmei Zu (China) Pei Zhang (China) Xiaohong Huang (China) Zhonghui Li (China) Chengwan Zhang (China) Anlun Long (China) Mohammad Tawrit (Bangladesh) Roshan Ragel (Sri Lanka) M Hussain Fageri (Afghanistan) Mohd Noh Jasmani (Malaysia) Dr A. Paventhan (India) Franz A. de Leon (Philippines) Shinji Shimojo (Japan) Louisa Lam (Hong Kong) Xiaoming Fu (Germany) Ning Wang (United Kingdom) Lochan Lal Amatya (Nepal) Simon Peter Green (Singapore) Chalermpol Charnsripinyo (Thailand)
Partner organization	AARNET(AU), APAN-JP, BdREN(BD), CERNET(CN), DOST-ASTI(PH), ERNET(IN), JUCC/HARNET(HK), ITB(ID), KREONET(KR), LEARN(LK), MYREN(MY), NREN(NP), PERN(PK), REANNZ(NZ), SingAREN(SG), ThaiREN(TH), TransPAC/GNA-G Routing Working Group/APAN Routing Working Group(US), abbr. APAN Routing WG). The 2 research universities are the University of Gottingen(DE) and Surrey University(UK)

Project Summary

BGP is one of the key infrastructures that allows the Internet to operate. This project aims at developing a collaborative platform for BGP routing services. The following are the main objectives and the main activities conducted during the reporting period of this project: 1. Build a collaborative community for enhancing the capacity of NRENs' network operation and measurement. 17 NREN organizations in the Asia Pacific region and 2 research universities in Europe participated in this project. They built the project governance structure and the teams with the responsibilities of all partners. This project is also collaborating with APAN, APAN Routing Working Group and APNIC.  The 17 NREN organizations are AARNET(AU), APAN-JP, BdREN(BD), CERNET(CN), DOST-ASTI(PH), ERNET(IN), JUCC/HARNET(HK), ITB(ID), KREONET(KR), LEARN(LK), MYREN(MY), NREN(NP), PERN(PK), REANNZ(NZ), SingAREN(SG), ThaiREN(TH), TransPAC/GNA-G Routing Working Group/APAN Routing Working Group(US, abbr. APAN Routing WG). The 2 research universities are the University of Gottingen(DE) and Surrey University(UK). 2. Establish a distributed BGP routing monitoring platform and a CGTF Looking Glass platform in the Asia Pacific region. The platforms are established, and 15 NREN partners set up the sessions with the BGPWatch platform and 7 NREN partners set up the connections with the CGTF Looking Glass platform. Functions include:  • Dashboard: AS info, prefix, peers • Routing path: forward, reverse, bi-direction • Subscribe and Alarming: AS prefix change, AS neighbor change, prefix routing path change 3. Deploy a BGP hijacking detection and mitigation system and analyze the robustness of routing in the Asia Pacific region. The system was developed, and one technical report and two research papers were published: • The report "Analysis of Suspected Hijacking Events of Internet Routing Prefixes in 2022"  • One research paper on "Metis: Detecting Fake AS-PATHs based on Link Prediction" published at ISCC 2023 • One research paper on "Evaluating and Improving Regional Network Robustness from an AS TOPO Perspective" published at NOMS 2023 4. Share our knowledge and experiences globally. Based on the partners’ requirements, the project team organized several events for knowledge and experience sharing: 4-hour online RPKI training including theory and hands-on training, 7.5 hours of physical training for RPKI and MANRS, 2 sessions for experience sharing of the deployment of RPKI and MANRS and one-day hybrid training for DNSSEC. Additionally, this project provided 24 enrollment for project partners to take the courses provided by CompTIA to enhance their capabilities of network security. The presentations of the project updates were delivered at the joint meetings with APAN Routing WG at APAN53/APAN54/APAN56 and at 2 open project meetings at APAN55/APAN56. An invited talk at APAN Routing WG monthly meeting, and a talk at the side meeting of TNC23. This project is also a part of the exhibition material of the physical booth at IGF2023. All these documents, two research papers and a 2022 report are open to all who are interested. This could have brought more benefit to the wider community in this region and globally. In addition to the above, we acknowledge the valuable contributions of APAN-JP, AARNET, REANNZ, and APAN Routing WG to this project, all of whom provided their support without receiving any project funds. Furthermore, we appreciate KREONET and SingAREN for their contributions to this project, partially funded by the project.

Table of contents

• Project factsheet information
• Background and justification
• Project Implementation Narrative
• Project Review and Assessment
• Capacity Building
• Gender Equity and Inclusion
• Project Management
• Project Sustainability
• Project Communication
• Project recommendations and use of findings
• Bibliography

Background and Justification

It is a known fact that there is a) no large-scale cooperative monitoring system for Border Gateway Protocol (BGP) routing and b) no collaborative system for BGP hijacking and mitigation among National Research and Education Networks (NRENs) in the Asia Pacific area.

1) Looking Glasses and BGP monitoring platforms can help operators monitor and diagnose BGP events. Looking Glasses are used for verifying routing between the providers and verifying that routes are propagating correctly across the Internet. A Looking Glass allows us to view the Internet from different locations. It can help network operators quickly locate network faults and provide network researchers with meaningful data. The BGP monitoring platform provides a tool for Internet operators to obtain real-time BGP information about the global routing system from the perspectives of several different backbones and locations around the Internet.

2) State of the art of BGP monitoring and Looking Glass platform 
RIPE’s Routing Information System (RIS) has 21 route collectors (RCs) distributed worldwide, collecting BGP updates from around 300 peering ASes. RouteViews provides control-plane information collected from 19 RCs that are connected to nearly 200 ASes worldwide. BGPmon from Colorado State University provides live BGP feeds from several BGP routers of the RouteViews sites, and a few dozens of peers worldwide. Well-known LG pages include PeeringDB, Traceroute.org, BGP4.as, and BGPLookingglass.com.

3) Active hijacking detection and mitigation is important 
BGP is a distributed protocol, lacking authentication of routes. As a result, an AS can advertise illegitimate routes for IP prefixes it does not own. These illegitimate advertisements propagate and “pollute” many ASes, or even the entire Internet, affecting the availability, integrity, and confidentiality of communications. Although RPKI and BGPsec are proposed, they can be fully efficient only when globally deployed, and operators are reluctant to deploy them due to technical and financial costs.

4) State of the art of hijacking detection and mitigation research
ARTEMIS (Automatic and Real-Time Detection and Mitigation System) is the state of art hijacking detection system. Before running ARTEMIS, administrators need to configure information such as AS neighbors, IP prefixes, and routing policies in advance, and then complete the detection by checking whether the routes observed by external observation points are consistent with expectations. The limitation of ARTEMIS is that the ISP needs to operate and maintain itself, and it cannot detect hijacking events of other ASes.

5) There is work we have already done
In the CGTF Looking Glass WG, a small-scale CGTF Looking Glass platform is under development, and a manual specifying how each partner uses it is provided. The Tsinghua team has helped the other NREN partners fulfill the configuration on their routers. In the BGP routing sharing WG, a small-scale BGP routing collection platform is under development, and now the ongoing actions are building the web and FTP service to share the data and writing a manual specifying how each partner uses it and how to configure their routers.

Our team participates in the operation and management of the CERNET network and has practical experience. We have both network management needs and development capabilities and this gives us an advantage in working with a wide range of partners. 
The project promoted cooperation through working groups and meetings among NREN partners in the Asia Pacific region and established the CGTF Looking Glass and BGP routing sharing platforms, a hijacking detection and mitigation system. These are very helpful for NREN network operation and measurement. 

Project Implementation Narrative

Problem Statement: BGP plays a pivotal role in enabling the smooth of the Internet. However, like the rest of the world, the Asia Pacific region faces a number of challenges related to BGP routing services, including network operation, measurement, security, and knowledge sharing. This project was initiated to address these particular issues. 

Project Objectives:

1.  Building a collaborative community to enhance the capacity of National Research and Education Networks (NRENs) network operation and measurement: 
The primary aim of this project was to create a collaborative community that would empower National NRENs in the Asia Pacific region to improve their network operations and measurement capabilities. 17 NRENs actively participated in this initiative. The following activities were undertaken: Collaboration with APAN and APNIC: To leverage expertise and resources, the project engaged with the Asia Pacific Advanced Network (APAN), APAN Routing WG, and the Asia Pacific Network Information Centre (APNIC). In addition to the eight NRENs which participated in the initial project supported by the Chinese government, these collaborations in this project attracted nine more NRENs to participate in this project which fostered a broader and more inclusive approach to addressing BGP-related challenges.  Learned and studied together: The Project Executive Team searched for materials from the Internet and shared them with all project partners. We studied together, discussed, did tests, and reached solutions that were suitable for our project partners. The project fully considered the needs of each partner by arranging bilateral meetings and trained NREN partners' operators and discussed the project activity plans together.  Governance structure and team building: The project began by establishing a governance structure, with clearly defined responsibilities for all partners and proceeded with the detailed work plans and regular meetings. This collaborative framework was discussed and supported by all project partners, which provided the foundation for effective project management and execution.  

2. Establishing monitoring and measurement platforms: 
To enhance network visibility and control, the project focused on setting up a distributed BGP routing monitoring platform and a CGTF Looking Glass platform within the Asia Pacific region. 15 NREN partners successfully established sessions with the BGPWatch platform, while seven NREN partners set up connections with the CGTF Looking Glass platform.  
These platforms provided real-time insights into AS information, prefixes, peers, and routing paths (forward, reverse, bi-directional), and enabled subscription-based alarming for AS prefix changes, AS neighbor changes, and prefix routing path alterations. 

3. BGP hijacking detection and mitigation: 
Security concerns related to BGP routing were addressed by developing a BGP hijacking detection and mitigation system. Additionally, the project conducted an analysis of the robustness of routing in the Asia Pacific region. The highlights included: Publication: A report was released that covered regional prefix hijacking events during the 2022 timeframe, shedding light on the state of routing security in the region. Furthermore, the project made contributions to academic research by publishing two papers at NOMS 2023 and ISCC 2023.  Platform development: The project successfully created the BGP hijacking detection and mitigation platform, bolstering security in the region's network infrastructure.  

4. Global knowledge and experience sharing: 
The project recognized the importance of sharing knowledge and experiences globally to benefit a wider community. To achieve this, the project conducted several activities:   Engagement and presentations: The project actively engaged with the community by participating in joint meetings with APAN Routing WG and open project meetings. We also delivered presentations at other related sessions and events, including four APAN meetings and TNC23 conferences. As part of the exhibition material of the physical booth at IGF2023, this project will introduce project outcomes to that audience.  Training and workshops: To help network operators, the project team gave talks on "How to use the BGPWatch platform" and also provided a demonstration video and manual to all project partners (and to any others who are interested). Meanwhile, with support from APNIC, the project team organized a series of events based on the partners' requirements. These included a 4-hour online RPKI (Resource Public Key Infrastructure) theory training and hands-on training, five physical training sessions for RPKI and MANRS (Mutually Agreed Norms for Routing Security), one-day DNSSEC hybrid training, and two sessions for experience sharing of the deployment of RPKI and MANRS. We also provided 24 enrollments of CompTIA online courses on network security for those project partners who required it.  
As the collaborative platform for BGP routing services, this project successfully addressed the pressing issues related to BGP routing in the Asia Pacific region. By establishing a collaborative community, deploying monitoring platforms, enhancing security, and sharing knowledge, the project made significant strides in improving the resilience and efficiency of BGP routing services. Their efforts not only benefit NRENs but also contribute to the broader Internet community. 

Project Collaborations and Partnerships: 
The success of the project was greatly amplified by strong partnerships with various organizations, researchers, and community leaders. These collaborative efforts played a pivotal role in achieving the project's objectives: 

1. Tsinghua/CERNET Team: The Tsinghua/CERNET team's involvement was instrumental in the project design and implementation. Their expertise and commitment were crucial in building the project's governance structure, collaborating with all project partner organizations, and conducting activities effectively. 

2. APAN (Asia Pacific Advanced Network): With the support of APAN, this project successfully invited more NREN organizations to participate and contribute to build a wider community in this region.  

3. APNIC (Asia Pacific Network Information Centre): APNIC's support and engagement were pivotal in shaping the project's direction. They actively contributed through knowledge sharing, training sessions, and project meetings. 

4. TransPAC (GNA-G/APAN Routing Working Group): Collaboration with APAN Routing WG, facilitated a broader and more inclusive approach to addressing BGP-related challenges. This partnership expanded the project's reach and impact. 

5. NREN Partners: The active participation of 17 NRENs from the Asia Pacific region contributed to the project's diverse perspectives and expertise. 

6. BdREN Contributed to BGPWatch Manual and Demonstration Video and investigation of CompTIA: The creation of a BGPWatch manual and demonstration video, a collaborative effort between BdREN and Tsinghua University, served as crucial resources for the project. The training courses from CompTIA were very helpful for NREN network operators. 

7. We acknowledge the valuable contributions of APAN-JP, AARNET, REANNZ, and TransPAC to this project, all of whom provided their support without receiving any project funds. Furthermore, we appreciate KREONET and SingAREN for their contributions to this project, partially funded by the project. 
The project was a true testament to the power of collaboration. These partnerships and the dedication of individuals and organizations across the globe were integral in achieving the project's objectives and advancing the field of BGP routing services. Their collective efforts have had a lasting impact on this project and beyond. 

Project Timeline: 
Feb. 24, 2022: Kick-off meetings were held with 13 partners and the new partners' invitation started. 
Partnership: 13countries/economies provided letters of support to this project proposal (see below)   
CERNET(China), ThaiREN(Thailand), SingAREN(Singapore), APAN-JP, HARNET/JUCC(Hong Kong, China), LEARN(Sri Lanka), BdREN(Bangladesh), MYREN(Malaysia), NREN(Nepal), ERNET(India), DOST-ASTI(PREGINET, Philippines), Gottingen University(Germany), Surrey University(UK)    
(ref: [APNIC Project] Kick-off.pdf)   

Mar. 10, 2022: Delivered a presentation at APAN Routing WG meeting at APAN53 to share the project information and plans for inviting new partners. 
(ref: [APNIC Project] Report at APAN53.pdf)   

Mar. - Apr., 2022: Invitations were sent and bilateral meetings were held with new partners and 6 new partners confirmed their participation, including AARNET(AU), ITB(ID), KREONET(KR), REANNZ(NZ), ERNET(IN), TransPAC(US, APAN Routing WG) 

Apr. - May, 2022: Bilateral meetings with all partners.  
Exchanged ideas and collected comments and suggestions on the project objectives and governance, coordination and management, roles of partner organizations and expectations, activity plans and funds policy, etc.   
There were also some technical concerns from partners, and some solutions were suggested accordingly, which can be found in the presentation of the first Technical Committee Meeting report.   

May 10, 2022: The first Joint Meeting of the Coordination Committee and Technical Committee was held. 
The outcomes from our bilateral meetings and summarized comments and concerns from partners were shared. This meeting also finalized the project governance structure and the activity plans. Each partner confirmed their representatives in both Coordination and Technical Committees. The Project Executive Team was set up mainly by the Tsinghua University team. It was decided that the Coordination Committee meeting will be quarterly, the Technical Committee meeting will be monthly, and the Project Executive Team meeting will be biweekly.   
The Coordination Committee was made responsible for project policy, strategy, project activity plan, monitoring project management and financial policy.   
The Technical Committee was made responsible for the technical activity plan, technical discussion of project development and implementation, and research papers/reports.   

The Project Executive Team was made responsible for project management, implementation, service/platform program development, engineering collaboration, communication and coordination of all partners and different teams, dissemination, meeting/event organization, documentation and financial works. 
(ref: [APNIC Project] 1st Coordination Committee Meeting.pdf, [APNIC Project] 1st Technical Committee Meeting.pdf)   

Apr. - May, 2022: The project website was set up, to be updated periodically.   
(ref: https://bgper.net)   

June 20, 2022: The second Technical Committee Meeting was held. 
The project progress was shared and the work plans for the next 3 months were proposed and discussed.   
Some functions of the BGPWatch platform were developed, including the dashboard function and Forwarding Routing Path Search. 
(ref: [APNIC Project] 2nd Tech Committee Meeting.pdf)   
(ref: https://bgpwatch.cgtf.net)   

Aug. 3, 2022: The third Technical Committee Meeting was held. 
The progress was updated including establishing BGP sessions with 9 partners and developing some new functions of BGPWatch, including Register, Subscribe AS, and Send Alarm Email to Subscriber. The next work plan was reported and discussed.   
(ref: [APNIC Project] 3rd Technical Committee Meeting.pdf)   
(ref: CGTF BGP RIS Platform Manual.pdf)   
(ref: https://bgp.cgtf.net)   

Aug. 25, 2022: Delivered the report at APAN54, jointly held with APAN Routing WG meeting.   
(ref: [APNIC Project] Report at APAN54.pdf)   

Sep. 23, 2022: Chairs meeting was held. 
There were some topics discussed and supported including solutions to the long-term plan, the current status of partners' engagement, the draft plans of collaboration and knowledge-sharing events in the next few months.    
(ref: [APNIC Project] Chairs meeting.pdf)   

Sep. 29, 2022: The second Joint meeting of the Coordination Committee and Technical Committee(4th) was held. 
This meeting summarized the partner's engagement, updated the progress of the 13 BGP sessions established and CGTF Looking Glass deployment plan, and the draft plans of collaboration with other organizations and the events for knowledge sharing.   
(ref: [APNIC Project] 2nd Coordination Committee Meeting.pdf, [APNIC Project] 4th Technical Committee Meeting.pdf)   
(ref: CGTF Looking Glass Platform Manual.pdf)   

Oct. 13, 2022: A meeting was held for a research paper discussion and the paper is to be titled Evaluating and Improving Regional Network Robustness from AS TOPO Perspective.   
(ref: [APNIC Project] Paper Discussion.pdf)   

Oct. - Nov., 2022: The second round of bilateral meetings with 16 partners was held.   
AARNET, APAN-JP, BdREN, DOST-ASTI, Gottingen University, HARNET, ITB, KREONET, LEARN, MYREN, NREN, PERN, REANNZ, SingAREN, Surrey University, and ThaiREN   
The following technical comments and suggestions were collected from the partners: Screen Resolution Auto Adaption.  Error when searching IPv6 address routing.  Statistics error on the Home page. Configure interested prefix/AS, and send an alert when anomaly/hijacking. More BGP-related alerts, such as peer change/path change. Send a message by Slack channel. Bi-direction routing path. Reverse routing path    Monthly /weekly summary. Show alternative routing path/track multipath. Path performance.  
The finance status and payment methods for each partner were discussed and agreed to. The following draft plan of knowledge-sharing online events in Nov/Dec was proposed and the comments were collected. 

Jan. 19, 2023: The third Joint Meeting of the Coordination Committee and Technical Committee (5th) was held.   Brief Summary of 16 Bilateral Meetings.    Discussed and decided on RPKI & MANRS Training Plan.    Established BGP sessions with 14 partners.  CGTF Looking Glass connected with 7 partners.    BGPWatch implemented a bi-directional routing path function.    The research paper was accepted by NOMS 2023, and the open source can be accessed at: https://github.com/thudragonlab/Resilience  Discussed Draft version of Prefix Hijacking Report in 2022.  
(ref: [APNIC Project] Project Executive Team Report.pdf, [APNIC Project] 5th Technical Committee Meeting.pdf)   
(ref: CGTF Looking Glass Manual.pdf)   

Feb. 1 & 3, 2023: RPKI online training. 
It drew 80 attendees from 19 countries and regions. 

Feb. 20, 2023: Release of Prefix Hijacking Report in 2022.   

Mar. 12, 2023: Release of BGPWatch manual and video.   
Manual for BGPWatch:   
https://www.cgtf.net/wp-content/uploads/2m023/07/User-Manual-for-BGPWatch.pdf 
Video for BGPWatch:   
https://www.cgtf.net/wp-content/uploads/2023/07/BGP-Watch-Video-V4.mp4 

Mar. 15, 2023:  Project Open Meeting at APAN55. 
Reported project progress, and did system demonstration. 
Established BGP sessions with 15 partners. 
BGPWatch implemented a reverse routing path search function with good interactivity. 
(ref: [APNIC Project] Report at APAN 55.pdf) 

Mar. 13 - 16, 2023: RPKI & MANRS Training at APAN55. 
7 in-person sessions, including: 
RPKI-Theory + Hands-On Training (4 sessions). 
A panel for RPKI User Cases and Experience Sharing. 
A MANRS training session. 
A panel for MANRS User Cases and Experience Sharing. 

Apr. 27, 2023: The sixth Technical Committee Meeting was held. 
Suggestions and comments were reviewed and some bugs were fixed.  
The work plan for the next four months was reported and discussed. 
(ref: [APNIC Project] 6th Technical Committee Meeting.pdf) 

May 24 - 25, 2023: An onsite meeting was held at Tsinghua University. 
Research topics included “Detecting Fake AS-PATHs Based on Link Prediction” and “Outsourcing Mitigation against BGP Prefix Hijacking”.  
In the DNSSEC training, over 60 Engineers and Technicians physically participated and online over 1000 participants listened to the training lectures. 

June 8, 2023: Presented the project at TNC23. 
Offered a comprehensive presentation on the project's goals, objectives, and progress, and received positive feedback at the side meeting at TNC23. 
(ref: [APNIC Project] Report at TNC23.pdf) 

Aug. 15, 2023: 24 enrollment of CompTIA online courses of network security for most project partners who required were completed. 

Aug. 24, 2023: The project concluding meeting was held at APAN56. 
Jilong Wang presented the Coordination Committee Report and summarized the project objectives and achievements, especially the work of building collaborative communities, sharing knowledge, the deliverables and dissemination, and the project budget and expenses. 
Changqing An gave the report of the technical teams which especially focused on the platforms development of BGPWatch and Looking Glass, the progress of the hijack detection and diagnosing system and shared the draft future work plans of the next phase of the APNIC Foundation ISIF Asia project. 
(ref: [APNIC Project]Coordination Committee Report at APAN56.pdf) 
(ref: [APNIC Project] Technical Committee Report at APAN56.pdf)

Project Review and Assessment

• To what extent has the project achieved its objectives? 

As evident from the "Project Implementation Narrative" provided above, we have diligently executed the activities outlined in the work plan and successfully achieved the objectives that were set out to accomplish. These achievements can be summarized as follows: 

1. Building the project partner community:   Initiated the project partner community through the successful organization of the first round of bilateral meetings involving 19 partners.  Finalized the project governance structure, clearly defining the responsibilities of two committees and one team.  Successfully nominated and confirmed members for both committees and teams and defined the mechanism of collaborative work.

  
2. Establishing regular meetings:   Organized a series of meetings, including biweekly Project Executive Team meetings, three rounds of formal bilateral meetings with all project partners, monthly Technical Committee meetings, and quarterly Coordination Committee meetings.  Effectively gathered and prepared relevant information and documents of these meetings for timely sharing updates, facilitating informed discussions on the work plans, and initiating the research work.  

3. Project website setup:   Launched and maintained the project website (https://bgper.net), ensuring it remains up-to-date and informative. 

 
4. Heavy work on the platforms' development:   Successfully established the BGP routing collection platform.  Established the CGTF Looking Glass platform, enhancing network visibility.  Developed a BGP hijacking detection and routing monitoring platform to enhance network security.  

5. Research papers and reports:   Contributed to the field by producing two research papers that were published at NOMS 2023 and ISCC 2023.  Authored a technical report on prefix hijacking, providing valuable insights into routing security.  

6. Comprehensive handbook and instructional video:    The development of an extensive BGPWatch handbook and an instructive video, achieved through collaborative work between BdREN and Tsinghua University, proved to be invaluable assets for the project.   The well-structured manual for RIS and the Looking Glass platform substantially aided other NRENs in establishing their own Looking Glass services.  

7. Knowledge sharing – Training events:   Conducted an RPKI Online Training event, open to participants beyond the project partners, which drew 80 attendees from 19 countries and regions.  Facilitated 7 physical knowledge-sharing sessions on RPKI & MANRS Training at APAN55, which drew 169 attendees.  Organized a successful DNSSEC Training event in Beijing, with more than 60 onsite participants and over 1000 online participants.  Explored CompTIA Certificates courses, resulting in 24 enrollments, supported by BdREN and approved by APNIC Foundation. 

 
8. Deliverables and dissemination:   Project presentations were delivered at APAN53, APAN54 and APAN56 in collaboration with APAN Routing WG, as well as at a project meeting at APAN55 and a presentation at TNC23 in partnership with the CERNET and GEANT teams. These presentations received favorable feedback.   A variety of posters and banners were distributed during both online and in-person knowledge-sharing events.  

In summary, the project has made substantial progress in achieving its objectives, leading to significant outcomes and knowledge-sharing activities that have had a positive impact on practitioners and researchers in the field. 

• What were the most important findings, outcomes and outputs of the project? What are your plans to use and promote them?   

Routing hijacking and monitoring are still concerns for most network operators.   
This project has helped build a collaborative community in the Asia Pacific region to work together on finding and solving routing-related issues.   
Three systems have been implemented: BGP routing sharing platform, CGTF Looking Glass platform, hijacking detection and monitoring platform.   
The platform is welcomed by partners and gets much positive feedback and comments from across the NREN community. The project outcomes have been presented and demonstrated at APAN55, APAN56, and TNC23, and will be shared at IGF2023 as well. Some new partners are expressing interest in joining this project.  
The project website was published and kept up-to-date. 

• What contribution to Internet development did the project make? Is there already evidence of positive impact?   

The project has made a substantial contribution to the development of Internet routing issues in the Asia Pacific region.  
The platforms developed through the project have provided NREN network operators with valuable tools to monitor their networks and facilitate information sharing within the broader Internet community. Notably, network operators have provided highly positive feedback, affirming that the project has already begun to enhance network functions and is poised to continue doing so in the future. This evidence underscores the project's positive impact on Internet development. 
The training and experience-sharing of RPKI, MANRS and DNSSEC, and CompTIA online courses for network security gave more instructions and experiences to those NRENs who are considering if their networks are ready to deploy these services to improve their network security. 

• To what extent has the project lived up to its potential for growth/further development?   

The partners have shown their strong interest in this project, and there are still many functions they wish to see in the future plans.   

• To what extent have the project activities supported the development of local technical capacity?   

Many engineers proposed requirements, evaluated and used the systems, and took part in writing the manuals.   
4 master students and 2 engineers from the Tsinghua team joined the software development and research work team. 
2 staff members gained experience in managing large international projects.  
RPKI, MANRS, DNSSEC training, CompTIA courses and knowledge sharing also supported the development of local technical capacity.   
 All these combined to support and increase technical capability across the Asia Pacific region.   

• What lessons can be derived that would be useful in improving future performance?

   
Communication and collaboration are keys to the success of the project.   
Regular project meetings and bilateral meetings are essential to enhance effective communication with all partners as more detailed issues could be shared and solved after those discussions.    

• To what extent has the project helped build the capacity of your institution or of the individuals involved?   

Discussion, experience, and knowledge sharing helped the following of the institutions and individuals involved in this international project.    
BGP routing knowledge and service development. 
International collaborations with other international groups in the BGP routing area. 
Network security-related knowledge: PRKI, MANRS, DNSSEC and CompTIA 
International project management training. 
The capability of communication with international partners. 
The capability of dissemination and event organization of this international project. 
English language skills. 

• Were certain aspects of project design, management, and implementation particularly important to the success of the project?   

Demand analysis and communication are particularly important to the success of the project. Only when the project is useful to those involved, will they truly be interested in it and contribute to it.

Capacity Building

The goal of this project is to collaborate with all possible NREN partners to enhance the capacity of NRENs' network operation and measurement in the Asia Pacific region.  The capacity building plan and implementation includes the following main activities: 

1. Capacity building in Tsinghua University team 
Usually, the international projects led by Tsinghua University have between 3 and 5 international partners – this project has 19 international partners. It is therefore the biggest international project led by Tsinghua University. In addition, the 18-month duration is another challenge to coordinate with more partners for daily project management, and event organization. The Project Executive Team is mainly from Tsinghua University to ensure more effective work for well conducting the project activities. There are the needs for the main members and engineers of the project executive team to provide training opportunities in international project management and some BGP-related knowledge-sharing events.   
- 2 members were trained for international project management training courses and passed.   
- 2 engineers who were supported by this project joined the physical training event at APAN55 (see below). 
- 10 engineers from China joined RPKI and MANRS training. 
-  About 60 engineers from the universities in China physically joined DNSSEC training in May in Beijing. 
- 6 CERNET engineers enrolled in the CompTIA online courses. 

2. Building the community of BGP Routing in the Asia Pacific region   
It is planned to invite NREN partners to join this project and contribute to developing the collaborative BGP routing-related platform. The main actions for building this community have been taken below:   
- March and April in 2022   
· Sending out the invitations and arranging bilateral meetings with all possible NREN partners for well understanding of the project plan, and confirming their participation in this project, besides the 13 organizations (11 NRENs and 2 research universities) that provided the letter of support in the proposal to APNIC Foundation, 6 more NRENs confirmed to participate in this project.   
- May 2022 to August 2023   
· Arranging regular meetings with all partners for the project development, implementation and coordination, including 3 rounds of formal bilateral meetings, quarterly Coordination Committee meetings, monthly Technical Committee meetings, bi-weekly Project Executive Team meetings, one meeting for research paper discussion, joint meeting with APAN Routing WG at APAN53/APAN54/APAN56, open project meeting at APAN55/APAN56, a presentation at APAN Routing WG monthly meeting, and a presentation at TNC23 (the European NREN conference).   

3. Three knowledge-sharing events have been organized successfully and very good feedback was received, and CompTIA courses are provided too. 
- Working with APNIC, two 2-hour online sessions for RPKI Theory and RPKI hands-on training were organized in February 2022. More than 80 training opportunities were provided, 6 females got 12 training opportunities. 
- Working with APNIC, APAN, NREN, the physical events were organized at APAN55, including full-day sessions for RPKI Theory and RPKI Hands-on Training, one 90-min session for MANRS training, one 90-min session for User Cases and Experience Sharing of RPKI Deployment, one 90-min session for User Cases and Experience Sharing of MANRS deployment, one 90-min session for Project Progress Update and BGPWatch Platform Demonstration. 169 training opportunities in 22 countries were provided. In addition, there was a female assistant trainer and 7 female trainees, with about 40 training opportunities provided to female trainees. 
- Working with APNIC, a one-day hybrid training for DNSSEC in May in Beijing attracted 60 physical engineers and over 1000 online local engineers. 
- 24 enrollment of CompTIA online courses in network security are provided for most project partners who showed interest.  

4. The Manual and demonstration video of the BGPWatch platform supported by this project were delivered at APAN55 and updated for the physical booth at IGF2023. 
Based on the partner's suggestions, the manual and demonstration video were prepared through a sub-contract between BdREN and Tsinghua University. This helps all NREN partners understand how to use the platform services well and improve the functions as well.   

5. Online and Face-to-face bilateral meetings have helped project partners demonstrate the functions of the tools and received more feedback and comments effectively. 
There were 2 rounds of online bilateral meetings and 3 rounds of face-to-face conversations arranged during APAN55, APAN56 and May Beijing meetings, to effectively exchange more information about the project and future plans with partners.

Gender Equity and Inclusion

• To what extent did the project support active participation from women and gender-diverse people both as part of the project team and communities served? How was gender equality and inclusion addressed in the project team structure and decision-making? Has the project allowed for a particular contribution to capacity building of women, LGBTQI+ or marginalized social groups based on other diversity criteria? 

In our organization (ie. Tsinghua University), there was a high level of female participation and leadership.  
One female serves as the co-chair of the Technical Committee and one female serves as the leader of the Project Executive Team. 
One female is working in the Technical Committee and two female staff members are working in the Project Executive Team. 
This project does not have any concrete criteria for a female contribution from other international partners, but it was encouraged and there are several female engineers who joined the Technical Committee, including REANNZ, TransPAC, LEARN and APAN-JP. There were also several female engineers helped organize the training events and joined the training courses. 

• To what extent did the project support people with disabilities, both as part of the project team and communities served?  
The project did not specifically consider this aspect. 

• To what extent did the project support language and/or cultural diversity, both as part of the project team and communities served? 
The project primarily used English as the main language for communication and most project materials, given its diverse group of NREN partners in the Asia Pacific region. However, despite the language focus, the project has played a role in promoting cultural exchange and understanding among Asia Pacific nations through its collaborations and interactions. While English served as the primary medium of communication, the project's engagement with diverse communities has contributed to a broader cultural exchange within the region. 

• Has the project inspired change inside your organization in relation to diversity, equity, and inclusion? 
Yes, it did. More females are now being encouraged in this project community, and more engineers and staff are encouraged to be more active in international collaborations.

Project Management

There are 19 NREN partners in the Asia Pacific region in this project. While this brings a lot of advantages in building a bigger community for BGP routing-related techniques and services, it also brings a lot of challenges for the project coordination and management within the 18-month period of the project. 
Quick action was taken in late February 2022 to organize an initial meeting with those partners who provided an early letter of support to the proposal. This allowed us to propose the draft idea of the project governance structure and project timeline. Meanwhile, the actions of inviting other NREN partners were begun, as well as forming the project committees and teams. 

The Tsinghua team took the main responsibility for the programmer work of all platforms and systems in this project, and also daily project management including administration, staffing, collaboration with other organizations, finance and so on. The international project coordination and implementation including the strategy discussion, policy-making, technical discussions of the platforms, and collaborations for knowledge sharing was taken by the Coordination Committee and Technical Committee, with which all NREN partners are involved. 

After having the bilateral meetings with the partners and receiving the confirmation of their representatives in both committees, the formal proposal of setting up the above committees with the names of the chairs and members was discussed at the first joint meeting of both committees.  The responsibilities of both committees and the project Executive Team were agreed upon and finalized (ref: page 12 of the presentation named [APNIC Project] 1st Coordination CommitteeMeeting.pdf). The governance structure was proposed based on the recommendations from all partners and was agreed (ref: page 11 of the presentation named [APNIC Project] 1st Coordination Committee Meeting.pdf). All partners will be involved in the decision-making process through both committees. 

After 18 months and working with 19 international partners, the daily project operation is quite busy. The Project Executive Team is formed by the Tsinghua team including 1 professor, 3 students, 5 engineers, and 2 staff members. The main team members meet biweekly (even during the COVID-19 pandemic period) and discuss all the issues of project management and administration based on the agreed activity plan. One professor leads 2 engineers and students focusing on technical development, problem solving, and technical documents, papers, reports and presentations. One senior engineer leads 2 engineers focusing on the platform implementation. Another engineer leads 2 staff and 1 engineer focusing on all internal and international communication and all the administrative works of meeting organization and project management. 

The internal approval process for receiving funds and making contracts and payments to partners and sub-contractors inside Tsinghua University is somewhat time-consuming and has taken longer than we expected. On occasion, the university has had to make some adjustments to the regulations of the university's international project management system, and this also requires more collaborations between different offices. From the administrative point of view, Tsinghua University tried its best to give all kinds of support to this project, including staffing, sub-contracts, project event organization, international travel and so on.  
At the same time, all these works strengthened the Tsinghua team's capacity for managing bigger international projects. 

Another big challenge is the effective communication with all international partners during the COVID-19 pandemic to manage and implement the project productively.  
After having the first joint committee meeting, the main finding was the difficulty of expressing themselves well during the meeting, especially for the NRENs from developing countries/economies and non-native-English language countries/economies. The most effective and honest way for good communication was organizing the bilateral meetings even though they brought more workload to the project Executive Team. More rounds of bilateral meetings were organized and more comments and suggestions were received, and more functions were expected to be developed on the BGPWatch platform with even more ideas for improving the hijacking monitoring and diagnosing system. From those bilateral meetings, the demonstration manual and video seemed like a very necessary idea. Therefore, the Tsinghua and BdREN teams worked together to prepare this before APAN55 and updated them for IGF2023. 

Knowledge-sharing is an important part of this project community. The main topic for the first event of knowledge sharing focuses on RPKI and MANRS which are BGP-related techniques. Because of the COVID-19 pandemic, the plan to organize a physical event at APAN54 in China in August 2022 was withdrawn in its early stages. The alternative proposal of an online event was started in September of the same year and proposed to the committee meeting in October. Based on the suggestions from partners, this proposal was suggested to work with APNIC and AARNET together.  Due to time conflicts and the Christmas holiday timeframe, this online event was finalized and postponed to February 2023.  The topic was RPKI only. Therefore, the physical event of RPKI & MANRS training and experience sharing at APAN55 was organized successfully by working with APNIC, APAN and NREN (local host). Some NREN partners gave strong support to these training events, especially CERNET, AARNET, REANNZ, CSTNET, China Telecom, CENIC, etc. Another training on DNSSEC was proposed and organized in May in Beijing hybridly.  

Before the project completion, we were trying to fully use the project funds to provide more training opportunities for project partners. Based on the investigation supported by BdREN and the support by APNIC, 24 enrollment of CompTIA courses for network security was provided to 24 engineers from most NREN partners who showed their interest. 

In addition to the above, we acknowledge the valuable contributions of APAN-JP, AARNET, REANNZ, and TransPAC to this project, all of whom provided their support without receiving any project funds. Furthermore, we appreciate KREONET and SingAREN for their contributions to this project, partially funded by the project.

Project Sustainability

• Has the project generated opportunities for future development, such as new funding from partnerships, sponsorships, investment, or other funding mechanisms? 
A proposal to APNIC Foundation for the next phase of this project has been successfully applied for and received approval and another proposal for additional Chinese government funding is underway. 

• Has the project helped to consolidate your organization and strengthen its credibility? If so, how? If any of the project activities will continue after the end of the grant, please describe how your organization is planning to support future developments. 
Yes. We held many bilateral meetings, technical committee meetings, and coordination committee meetings, and presented the project at APAN53,  APAN54, APAN55, APAN56 and TNC23, and we got a lot of compliments.  
The project has helped consolidate our organization's position and enhance its credibility significantly. The project's activities have garnered substantial positive feedback and recognition from partners and operators. Engineers and partners have expressed that the project's work is of high quality and has the potential to provide substantial assistance in their day-to-day operations. 
While the project activities have received positive acclaim, we are planning to continue and support future developments beyond the grant's end. The system's functionalities will remain accessible to the wider community, and we will actively engage with engineers and partners to ensure its ongoing improvement and relevance to their needs. This sustained commitment to the project's objectives will further solidify our organization's credibility and contribution to the community.

Project Communication

As mentioned above in the Project Management section, effective communication with all international partners is a big, ongoing challenge, especially during the COVID-19 pandemic. It was our experience that everyone worked hard to maintain the connections that had been established,(for most partners), in pre-pandemic and even post-pandemic times. When the committees and teams were formed, the main working channel was the mailing list and the main information sharing platform was the project website. However, within two months we set up a Slack channel for more timely communication and this was very popular.  All the information that needed to be shared with partners was sent out via the mailing list and the Slack channel. This included the meeting announcements, meeting materials/reports/minutes, research papers, technical reports, and event pictures.  Most project information was uploaded to the project website – this included project and partner information, tools, documents, news/events and so on.

One of the outcomes of the first online joint committee meeting, the main was difficulty some found expressing themselves well during the meeting, especially for the NRENs from developing countries/economies and non-native English language countries/economies. The most effective and honest way to address this was to have more frequent and deeper discussions by organizing online bilateral meetings, even though this resulted in a greater workload for the Project Executive Team. The second and third rounds of bilateral meetings with 16 partners were successfully organized and more comments and suggestions were received.  The reality is that face-to-face meetings are nearly always the best way of communicating, especially when working with speakers of multiple languages. When the pandemic situation became better, more rounds of in-person bilateral meetings were held during the APAN55, APAN56 and May Beijing meetings. When most partners met in person at those meetings, the Project Executive Team found every opportunity to talk to the project partner in person and exchange ideas for improving the services and sharing concerns.

With regard to the project dissemination, besides the project website, project presentations have been also given at APAN53, APAN54 and APAN56 by collaborating with APAN Routing WG, and at TNC23 by working with the CERNET and GEANT team, and this project is also a part of exhibition material of the physical booth at IGF2023.  A range of posters and banners were distributed during the online and physical events of knowledge sharing. 

Project recommendations and use of findings

In the course of the project, we have gathered valuable findings and lessons learned that can be beneficial to other practitioners and researchers facing similar challenges or implementing comparable solutions. Here are our recommendations and insights:   

Enhanced communication: Over-communication is often underestimated. More frequent and in-depth discussions are essential for the success of projects in this domain. Organizing regular online bilateral meetings or discussions with the leaders and network operators of all project partners will facilitate active dialogue and establish a continuous feedback loop, which can ensure everyone is well-informed and aligned with project goals, and can fine-tune solutions and maintain relevance.   

Actively engage with network operators: It is imperative to engage actively with network operators. Listening to their feedback and understanding their specific needs is crucial. Initiating proactive discussions with operators can lead to more tailored and effective solutions.   

Collaborative planning and collaboration: This is vital for managing the project with the right approach. The collaborative work with practitioners and researchers in making decisions on strategy and activity plans will ensure that all parties are on the same page and the project aligns with the actual needs of the community it serves.   

Timely sharing of documents and lessons: Timely sharing of knowledge and experiences can lead to better-informed decisions and more effective solutions. This certainly benefits the wider community too.    
Dissemination is always necessary for the success of the project.   
Overall, our project has demonstrated the significance of proactive communication, engagement with leaders and network operators, and collaborative planning. We hope these recommendations and findings can be helpful for others working on similar projects.

Bibliography

[1] URL for project website: https://bgper.net 
[2] URL for BGP routing sharing: https://bgp.cgtf.net 
[3] URL for Looking Glass: https://lg.cgtf.net 
[4] URL for BGPWatch: https://bgpwatch.cgtf.net 
[5] Manual for BGPWatch/CGTF Looking Glass/CGTF RIS Platform:  
https://www.bgper.net/index.php/document/ 
[6] Video for BGPWatch: 
https://www.cgtf.net/wp-content/uploads/2023/07/BGP-Watch-Video-V4.mp4

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License