TECHNICAL REPORT
| Grantee |
Boom! Inc
|
| Project Title | Deploying IPv6 in Yap FSM |
| Amount Awarded | USD 110,000 |
| Economies where project was implemented | Micronesia (FSM) |
| Project leader name |
Lubuw Falanruw
|
Project Summary
Boom! Inc. was previously allocated a /23 IPv4 and a /32 IPv6 address block. Half of the IPv4 address space has been exhausted through the design of the internal core infrastructure along with deploying several key customer networks which have expanded dramatically.
Initially, Boom! had authorities/consultants saying that the submarine fiber spur that had been installed between Yap CLS (cable landing station) and the other end (which connected in Guam neutral zone CLS and had a 10Gb port with IP service) and all related infrastructure had been completed and was ready for use. A large World Bank grant had been used to accomplish this. This meant we could primarily focus on Yap fiber CLS plug and play and building our our Yap terrestrial network and services.
However, when we finally decided to take up the challenge of building our infrastructure, we discovered there was no infrastructure available at either end of Yap or Guam. This led us down an expensive, complicated path of not just building out our Yap internal core infrastructure, but also required that we build the 2nd internal core infrastructure in Guam which was even larger and extremely complex.
This exhausted the IPv4 address space. When we received this IPv6 grant, we knew it’s importance to our services moving forward, but later on the complexities between two core networks in two separate locations added to theIPv6 R&D workload and IPv6 grant deliverables.
We were fully aware that IPv4 is limited and the continued use of IPv4 will not only limit our growth, but will severely impact our ability to expand future services. As a result, we have made progress moving forward and adopting IPv6, and have implemented and tested it within our core.
During this effort, IPv6 activity is limited to our Internet backbone nodes with the address block advertised to both upstream carriers via eBGP. As a result, we have already fully committed to moving forward and adopting IPv6, not just for existing deployments, which have increased in scope, but for all future services.
At the present time, this project is focused on the design of the backbone infrastructure, IPv6 address allocation, and best practices for small to large scale customer deployments. Boom! Inc.'s enterprise lab network continues to be the area of focus for all testing and design needs. Several key customers, whom have been with us throughout the initial Internet services rollout, continue to have interest and excitement to be a part of this next phase.
Table of Contents
- Background and Justification
- Project Implementation Narrative
- Project Activities, Deliverables and Indicators
- Project Review and Assessment
- Diversity and Inclusion
- Project Communication
- Project Sustainability
- Project Management
Background and Justification
The National Government of the Federated States of Micronesia obtained a large World Bank grant to promote digital development. To meet World Bank requirements, the FSM passed the FSM Telecommunications Act of 2014. Among other things, this act requires fair, open access to broadband speeds to multiple telco operators. The FSM National Government also created a number of National Authorities to oversee and implement telco development and hired numerous outside firms and consultants. Among other things, these National Authorities and Consultants were to provide “plug and play” connectivity to multiple telco companies to the submarine fiber optic cable that reached Yap in August of 2017. iBoom was the first telco entrant and we found many gaps and blockages throughout our journey. There was mis-information about infrastructure that we were told was in place, only to find they were not. We had to fill in those gaps at our own cost. Despite all the National Authorities and outside firms and consultants that they hired, we were on our own, and unfair practices continue with National funds and the National Incumbent at political levels.
We were faced with two types of monopolies; the incumbent and a top heavy “system” where valuable resources are spent and never seen the ground, or used to support those actually in action servicing the unserved. This forced us to be extremely resourceful. In addition, in the rush to block us, and we can only assume the rushed incumbent network upgrade somehow caused a major crisis for our entire state, service went from bad to worse.
Project Implementation Narrative
We have already requested and been allotted an IPv6 block. After successfully applying for the ISIF Asia IPv6 grant, we have begun the planning of a full migration to IPv6 for our network. We have explored three project approaches, and the latest one is working off of a turnkey as a service platform. Our approach is to learn that type of service delivery, and see if we can build our own so we are not dependent.
- We have initiated the IPv6 services to be deployed beyond or behind our backbone nodes, but additional support is needed, through APNIC resources.
- We have completed the design of the backbone infrastructure, IPv6 address allocation, and have planned out best practices for small to large scale customer deployments.
- We have completed Boom! Inc.'s enterprise lab network to be the area of focus for all testing and design needs.
- Several key customers who have been with us throughout the initial Internet services rollout, have also expressed interest and are excited to be a part of this next phase, and that progress is underway. First round of “pilot sign ups” are completed.
The Challenges
The service dropouts from the incumbent meant our team was literally being hunted down by our local government offices to get connectivity. No contracts, no billing, etc. Just desperate needs. The incumbent had everyone on long term contracts, and we did not feel right to double bill. This created a state wide emergency, and changed what was supposed to be an initial smaller scope proof of concepts, to becoming over night a mission critical operator with giant demands.
In addition, we were forced to take on the massive challenges of building not just one destination network, but two core infrastructures (Yap and Guam). Our head engineer indicated that we had exhausted our IP addresses on just our two core infrastructures and giving out dedicated IP addresses, especially for commercial accounts.
We were certainly feeling the struggle of providing IP addresses. Our engineer was understandably holding tight controls. These circumstances, with the addition of our exposure and dealings with APNIC and the Foundation, helped us to educate ourselves on behalf of our client stakeholders regarding how critical the adoption of IPv6 had become. Our island nation’s IP needs have over the years gone from critical to an emergency and blocker. We owe much to the foresight of our head engineer and APNIC's further foresight, and education. After learning this knowledge we applied for an IPv6 grant.
We knew at the time while still building out our core infrastructure, that the best time to adopt IPv6 would be as early as possible, before our network and services grew larger, which then would only increase our scope of work and resources needed, and by taking advantage of being a green field.
There were major factors that drastically changed our original proposal and our actual achievable scope of work adjustments and what many would say put us on a path of failure which we pulled through but still plagues us today.
- With our entrance into the Telco/ISP space in FSM, where the incumbent monopoly was and still is part of FSM National government, we faced a lot of resistance. This was despite our Nation’s Act 2014 Laws that permitted private industry in this space - and which were a requirement of World Bank grants. The incumbent and only service provider at the time was on very old, antiquated infrastructure and was rushing to upgrade from old VSat to Fiber mixed in with old copper lines.
- The quality of Telco services for Yap, especially mission critical Government needs, Hospital, etc, had issues. Unfortunately, while we were still building out our network hoping to have a small controlled initial foot print that could have the time to harden, incumbent services went from bad to un-usable. Services became worse than they were prior vSat infrastructure. This created many emergencies, and we literally had offices and people running out to the iBoom team still laying fiber, and desperately asked for expedited hook ups. The iBoom team was praised at how fast they adjusted, brought services back online in a fraction of the time while the incumbent was still troubleshooting.
- Our team was split in two. All of this happened during COVID-19, and FSM borders were shut down. our 100% local capacity ground team was executing to meet emergencies, but our remote team, while very passionate, were in different time zones. This made it hard for the ground team and remote PM/Admin to stay in sync, and some were still learning how to properly manage scope of work, project activity administration. This meant scope of work had a tendency to drift. While this was going on, communications with the ground team would often be cut off, ending in the unfortunate, final budget paperwork submitted that ended up cutting our grant from the much needed original approved $250K, down to only $110K.
- Our original main engineer, who pulled of miracles with the team became extremely in high demand, and it was not sustainable for us to compete with his shrinking time available. He has on many occasions gone above and beyond. However, Boom took all necessary steps to find additional engineering talent to recruit, ending up going through three failed attempts, and finally landing on a good fit. In addition, since then, we were able to make arrangements with our original lead engineer starting again this month – with more consistent availability. However, we had no official agreements, paperwork to help cover our costs while their contracts were locked into long term contracts, and within a very short amount of time, we went from small hand selected accounts for QA/testing, to live in production accounts that grew five times what felt like over night. This caused our scope to increase drastically, almost to the point of paralyzing our IPv6 plans back to re-planning in terms of both costs and timeline.
Hence the major gaps, scope of work changes, and the budget cut from $250K to $110K.
We have become the first in the region to have started to migrate our entire network from IPv4 to IPv6. The biggest issue that the world is facing is a struggle for enterprises to migrate away from IPv4 to IPv6. In fact, judging from the incumbent Telco monopoly's responses to dedicated IP requests, they have run out of IPv4 and can no longer issue them. iBoom/Boom! Inc. is sprinting to deploy IPv6 live (again, descoped for this grant project). We are migrating fully to IPv6 and we will soon showcase its benefits to the rest of Micronesia and the world so that others migrate as well.
Project Activities, Deliverables and Indicators
Beginning of Project
| Activity | Description | #Months |
|---|---|---|
| Core Infrastructure IPv6 Deployment | Work with experts to design and deploy IPv6 addressing to our existing backbone nodes | 2 |
| Boom! Inc. IPv6 Lab | Build a lab network that is flexible enough to test and deploy IPv6 services (Firewall, Router, Switches, Wireless AP, laptops, and computers). | 1 |
| IPv6 Use Cases | Construct use cases for typical customer deployments | 1 |
| IPv6 Address Design and Allocation | Work with experts to determine IPv6 schemas along with acquiring tools to properly manage its use. | 2 |
Middle of Project
| Activity | Description | #Months |
|---|---|---|
| IPv6 Lab POC | Deploy and test IPv6 services within Boom! Inc's IPv6 Lab. | 1 |
| IPv6 Migration - Boom! Inc. Enterprise | Migrate IPv6 into Boom! Inc.'s existing enterprise infrastructure. | 1 |
| IPv6 Migration - Existing Customers | Migrate IPv6 into existing customer networks. | 2 |
| IPv6 Migration - New Customer | Deploy a new customer network limited to only IPv6 (IPv4 excluded). | 1 |
End of Project
| Activity | Description | #Months |
|---|---|---|
| IPv6 Migration - Private LTE Testing | Test IPv6 with new service offerings. At this time, we are exploring private LTE as the underlay, to provide Internet services throughout the island. Addressing will be restricted to only IPv6. | 3 |
| IPv6 Migration - Private LTE Limited Deployment | Deploy IPv6 to a limited set of customers. Design should allow for scalability and growth. | 1 |
| IPv6 Migration - Private LTE Deployment | Deploy IPv6 in a private LTE production environment. | 1 |
Throughout the Project
| Activity | Description | #Months |
|---|---|---|
| IPv6 Peering with Upstream Carriers | Work with our upstream carriers to assure that our IPv6 address space has been advertised correctly. Also, look into acquiring tools to monitor our address space to assure its integrity. | 1 |
Key Deliverables
| Deliverable | Status |
|---|---|
| Obtain IPv6 Block: 2023 Q2 | Completed |
| Internal Audit (Network readiness Assessment) Full detailed list of hardware and its IPv6 compatibility: IPv6 ready network components, components needing upgrades to support IPv6, components needing to be replaced to support IPv6. | Completed |
| Create a fully functional IPv6 test lab environment in which to activate the IPv6 address-family within the iBGP infrastructure. Build the lab environment to simulate and test a specific use-case customer deployment. | Completed |
| Develop IPv6 Allocation Plan. Determine how subnets will be determined & divided. Determine a plan in which IPv6 addressing is easy to aggregate thus reducing the routing table size and reducing impact on router performance (effective and logical). | Completed |
| Additional IPv6 Full Time In-house Engineer, and Trunkey Platform Transition (Dual Stack) to IPv6: Fully deploy IPv6 across the entire Boom! Inc’s enterprise network (wired/wireless). | Completed |
| Migrate all clients to IPv6: 1) plan the IPv6 migration for the existing customers; 2) activate IPv6-only for new customers and deployments | Descoped |
| CEO Socializing & Sharing with community about IPv6 Grant | Completed |
Key Deliverables - Detail
| Deliverable: Obtain IPv6 Block: 2023 Q2 Status: Completed Start Date: August 1, 2023 Completion Date: October 1, 2023 Baseline: Did not start or update progress report until now, due to actual work scope expansion. Activities: • This was performed by our original lead engineer • R&D in Solar Winds IP Manager – On Premise vs Cloud (did not discover till later) Outcomes: Received /32 IPv6 Address Block |
| Deliverable: Internal Audit (Network readiness Assessment) Full detailed list of hardware and its IPv6 compatibility: IPv6 ready network components, components needing upgrades to support IPv6, components needing to be replaced to support IPv6. Status: Completed Start Date: August 1, 2022 Completion Date: September 1, 2023 Baseline: 2 Dell Servers Purchased and Installed and Configured, with Solarwinds IP Manager installed and configured. Activities: • R&D • Collaborations, Meetings, Schematics between lead engineer and tech team (primarily Robert out of Bay Area, Pj, Jessy, and Jack • Documentations Created (documents were uploaded) • 2 Dell Servers, On Premise IP Manager Server Software orders placed Outcomes: First Internally Approved Project Plans, Procurement Lists, and Documentations (including existing network at the time, specific IPv4 spreadsheet/use/locations and planned migration to IPv6). |
| Deliverable: Create a fully functional IPv6 test lab environment in which to activate the IPv6 address-family within the iBGP infrastructure. Build the lab environment to simulate and test a specific use-case customer deployment. Status: Completed Start Date: August 1, 2023 Completion Date: December 1, 2023 Baseline: Lab Environment Created Activities: • Simulation both IPv4 and IPv6 on sub-section isolated network, but Dell servers within Yap side core infrastructure – mainly testing Solar Winds in staging environment – purposely did not go into live production environment. • Lead engineer provided some preliminary training with rest of tech team. Staging environment for trial and errors. Outcomes: More familiarity, and hands on practicing in staging environment (again, closed internal network). Goal was to go from R&D theoretical, to gaining more confidence by Tech team who have never done this, and also discovered changes/tweaks that will be needed to Core network infrastructure (production live environment). |
| Deliverable: Develop IPv6 Allocation Plan. Determine how subnets will be determined & divided. Determine a plan in which IPv6 addressing is easy to aggregate thus reducing the routing table size and reducing impact on router performance (effective and logical). Status: Completed Start Date: September 4, 2023 Completion Date: February 1, 2024 Baseline: Allocation has begun. Activities: Senior Engineer Time Spent and allocated, while having other local staff shadow. Outcomes: Lead engineer, tech team timesheet, and failed attempt to fly Jack engineer out. |
| Deliverable: Additional IPv6 Full Time In-house Engineer, and Trunkey Platform Transition (Dual Stack) to IPv6: Fully deploy IPv6 across the entire Boom! Inc’s enterprise network (wired/wireless). Status: Completed Start Date: August 5, 2024 Completion Date: August 7, 2024 Baseline: No IPv6 in use Activities: New Hire completed and fully onboarded and in action. Switched to turn-key full service support 3rd party platform we found & completed vetting and due diligence, while also hired new Telco/Wireless/IPv6 migration engineer full time in-house, while our new head engineer continues to build out our own cloud infrastructure. Outcomes: • Switched to turn-key full service support 3rd party platform we found & completed vetting and due diligence, while also hired new Telco/Wireless/IPv6 migration engineer full time in-house, while our new head engineer continues to build out our own cloud infrastructure. This was the fastest track to deploy IPv6. • We completed our cloud own similar full service platform, and can resell it as a service to anyone else. Multi-tenant. Boom! Inc. has to pivot and evolve to being a service provider who is no longer held back/limited to National FSM. |
| Deliverable: Migrate all clients to IPv6: 1) plan the IPv6 migration for the existing customers; 2) activate IPv6-only for new customers and deployments Status: Descoped Start Date: August 7, 2024 Completion Date: September 30, 2024 Baseline: No Clients using IPv6 Activities: Descoped for the purpose of this grant, and budget limitations. However, we still have this as a soon to be "Ongoing" activity, one client at a time. Outcomes: Ongoing, one client at a time. |
| Deliverable: CEO Socializing & Sharing with community about IPv6 Grant Status: Completed Start Date: October 1, 2023 Completion Date: July 31, 2024 Baseline: Activities: Outcomes: No one knows anything about IPv6, nor IPv4, and why its important. Meetings with clients, stakeholders, and content write up about IPv6 importance for goal of "smart digital island" with smart devices integrated within island cultural, social, ecology, and environment applications and reasons why IPv6 is important vs IPv4, and PR/Educational materials. Secondary meetings/communications with local leaders to demonstrate our progressive progress, in simple terms, but critical for ongoing support by local stakeholders, and as role models for others in our region. Additional Comments: • CEO socializing, educating key stake holders and why its important. Meetings with clients, stakeholders, and content write up about IPv6 importance for goal of "smart digital island" with smart devices integrated within island cultural, social, ecology, and environment applications and importance of reason's why IPv6 is important vs IPv4, and PR/Educational materials. Secondary meetings/communications with local leaders so to demonstrate our progress, in simple terms, but critical for ongoing support by local stakeholders, and as role models for others in our region. Shared during three meetings with government executives and cabinet members. Again, shared as just one item on agenda, with meeting with Legislation. Draft creation of a one pager press release. On hold till actual deployment to at least one client. Outcomes: While within this report I only mention IPv6, the goal for socializing IPv6, was part of many other activities I felt we needed to start sharing things coming, vs. waiting until completion. The goals have two components here: Because National politics have not been cooperative, our local leaders have asked us to provide a report so they can draft a resolution to the national authorities. It’s important they understand the bigger picture; our work is far beyond just being an Internet connection & telco. The desired outcome was accomplished because they understand the application's importance – most of them don’t care about speeds. E-mail for example is a acceptable glorified snail mail. Concepts of real-time inter-departmental chats (slack for example) is a new concept. They are excited, but treading carefully. |
Project Review and Assessment
- One important lesson we have learned; everything takes a village. If there are bad weeds in the village, its probably better to first pluck those weeds out and deal with any red flags upfront. This may sound like a social lesson, but, it applies to technical achievements too. Some of our IPv6 challenges and unnecessary delays & costs could be tracked from the beginning with the system we started with. Technical bloat can rear its head at any point down the line.
- When presenting, our cultural leadership look past the subject you are explaining, as they are more interested in reading between the lines, and their main goal is watching your integrity, and making sure your intentions are transparent. As one of my chiefs said to me after several meetings; “so will this be good for us or not?”, I said “yes”. His response was “then get it done”.
- We had some challenges during our progress, as our Project Manager had to transition to other work, and our main lead network engineer, after doing extensive R&D on his own, was also met with extremely high demands over his time availability. We also have since hired more local on the ground talents, with knowledge transfer, onboarding, and a larger team. This took up lots of time. However, with our team grown to now over 25 locals on the ground, we have selected the lead PM for the rest of the work. He will be working closely with APNIC resources and the technical lead contact on closing up this project.
- The scope of our network also went through considerable expansion, leading to more work on designs, and a larger scope to fulfill IPv6 throughout a much larger user base and network topography.
IPv6/IoT side work
- During the project, the incumbent has run out of IPs or at least has stated to their clients (who wait typically 6 months to 8 months for expensive dedicated IP requests) they no longer have IP addresses to give out. Our ability to fast track IPv6 is it makes us very relevant, and sets us apart in a way where leadership at all levels, see that this is not just about one sector in a developing country. It helps broaden their view from seeing this as an old school telco. We are trying to push beyond the near-sighted "we need competition” attitude, to see that there is more than just this model - charging locals who have no economic capacity isn't a sustainable economic model. Applications demonstrate innovation and show what we are doing actually crosses over to every other sector. The current view is as if its just one sector.
- When our CEO was socializing our work about broadband connectivity, bringing in expanded FWA into communities, and IPv6, the method of explanations was less technical, and more about local relevance and impacts. He was then met with not just excitement and praise from community stakeholders but also questions and immediate community concerns and requests that were understandably of high priority & importance. Many of these community concerns related to climate change and sea level rise that is already having direct impact on our local food production systems such as taro patches being inundated with salt water.
- While no promises were made, IoT applications came into the picture, and as he usually does, our CEO undertook R&D to see how our IPv6 designs could accommodate devices that can monitor parameters such as soil conditions, salinity, temperature, and other factors that can be used to manage food production systems and marine resources on a sustainable basis.
- For example, our CEO is working with his mother, who is the authority in the region on indigenous and nature-integrated food production systems, to develop monitoring systems for salt water intrusion in taro patches via IoT so as to give communities an upper hand in taking proactive action to save these valuable food production resources. As this project is out of scope, and our CEO is stretched thin, this project was put on hold, in order to focus on grant scope completion.
- This is related to our IPv6 related scope about socializing, education, and spreading knowledge about the importance of the grant and IPv6 vs. IPv4. For illustration purposes, the CEO talked about running out of IPv4 addresses and how this will impact future mission critical needs and opportunities, such as use of IoT, in tracking future climate change impacts and food production securities on a fragile island. In this example taro patches were used. IoT applications here would require far beyond what IPv4 could support. This helps our stakeholders and people understand the importance of a technical issue like IPv6. There are over 2,000 taro patches in Yap and a need to deploy IoT to monitor and better understand how to deal with salt water intrusion into these food production systems. This work is relevant to other Pacific Islands and coastal areas of the world.
- The national borders also posed more challenges, as they would open and then close, which took place a few times. Things are stable now, as it relates to borders.
- CEO has shared and illustrated in simple visuals how the Importance of having enough IP’s to support the IoT example is also needed to help monitor fish populations in an ongoing innovative marine management program. Again, for the purpose of our grant scope, these activities was only to meet one of the key Deliverables: [Deliverable: CEO Socializing & Sharing with community about IPv6 Grants]
IPv6 Getting Back on Track
- Meanwhile, progress was made with procurement of additional gear (another Dell server, and SDWAN rackmounted gear), installed onto our datacenter racks, configured, IP management software was selected and installed. This also increased our costs, with expanded network, and need to create one server for lab work & testing, and another for production.
Wins so Far
- While our project fell short on some areas, on proactive project management, scope growth, and increased cost, and detailed budget submission was grossly under budgeted, and lack of proactive communications on our part with APNIC Foundation for guidance, we still had some takeaway progress wins so far. While we are under budgeted with expanded scope of work due to circumstances we could not control, but failed to document and submit a corrected budget; we remained committed, within our capacity, to complete the scope of work as identified within this IPv6 grant.
- Some highlighted wins in progress so far include the initial community awareness raising. This was met with increased understanding and connections of relevance specific to island community needs. We hope to find funding later upon completion of this grant, to address these needs through the use of IPv6 and IoT applications.
- We will leverage that relevancy, and other factors with the remainder of project as we seek to deploy IPv6 to our expanded scope of communities, stakeholders, and clients.
- We have added more local resources, creating more direct visible local capacity development, with our above and beyond work in parallel with both our FWA/mobile wireless work which has gone past scope and can be completed upon final detailed report and our IPv6 project progress.
Diversity and Inclusion
Our staff is multinational, multicultural, and works multilingually. We have a mix of both men and women and age groups. Our company policy is to ask, regardless of employees hiring role or position and regardless of gender, what other parts of our operations they may be interested in being involved in, either as shadow or direct hands on, pending training. This provides increased interest and understanding of our work.
Project Communication
We plan to primarily use the Internet and social media to promote our project developments. We have already done so with extreme success in past projects we have deployed. We focus on the power of our indigenous people to plan and deploy IPv6 across our island. We plan to continue to showcase this by capturing all stages of deployment and showcasing it on main stream social network platforms. We will engage our community with a website presence that allows for feedback and other forms of direct communication with those in the field that might require assistance with similar deployment. We will document all the various aspects of our progress from the beginning of this deployment in order to encourage others to follow suit. We will post on YouTube and create a section of our website for questions and answers for engagement. We will also feature our deployment plans and progress at an innovation hub in the heart of our island. All ages and genders will be welcome and we will give open access to all at our public hotspots. We will also work with traditional media journalists who cover the region and provide information on the status of our project, both our successes and our challenges. We will encourage our community members to take part in the development of the digital economy that will be built on the foundation we are laying with this grant.
Project Sustainability
Working with a local team helps promote self-reliance and our project’s sustainability. All aspects of upkeep and maintenance of the network through the migration and deployment of IPv6 has been handled by our ground tech team with remote oversight when needed.
When the incumbent Telco services suffered, stakeholders rushed us for servicing and we had to expand our network dramatically. This was out of our control, and grossly increased our scope of work. Now with a larger network user base that will have to be reconfigured with IPv6, our team will be taking advantage of APNIC resources to assist us with new staff training, and help us best address the expanded scope costs and best practices to bring closure to our circumstances and lPv6 services to our area.
Project Management
Another major factor that contributed to stretched out delays, cost increase as well as inability to complete the original objective to deploy our IPv6 was having access to professional engineers. Roughly a year plus ago, due to high demand of our original part time BGP & Core Gear/Router network Engineer, Robert, who in the beginning went from almost full time to very limited ability. Much of our core network access was understandably limited to him, due to security concerns etc. He had already gone above and beyond since beginning, and we could no longer compete with his compensation, despite it being heavily discounted for us. It was no longer sustainable for him or us. Despite this, he still continues to this day to help whenever he can, but it has been touch and go.
Also around this time, our CEO started head hunting for another resource while our local resources were doing their best in self training, and courses we purchased from Udemy. We went through three candidates and a lot of time and resources, until we found Jack Nahid, from Bangladesh. He is an amazing fit with both a technical skillset and also a cultural fit with the company. He was a big fan of iBoom, having looked us up. After 6 months of touch and go knowledge transfer between Robert and him, we finally decided to fly him out to Yap island to be on site. Time-zone wise this would be easier with Robert's schedule, and also being on site makes the world of a difference. Getting him on site turned out to be a challenge. In addition, during these challenges, our CEO was able to make new arrangements with Robert, to where now he is on sustainable retainer, and set scheduled availability.
Jack’s first trip attempt was canceled last minute due to typhoon that hit his area. The next two trips were also pushed back 1-2 months at a time due to the crisis in Bangladesh:
We had already invested in time and costs with Nahid. He is a expert in IPv4 migrations to IPv6, and we switched to social support, but did our best to balance work and deal with Nahid's lockdowns and the country's Internet shutdowns. Ethically we felt that we could not abandon our new team member, and seek a replacement.
We had many scheduled meetings between our teams, Robert, and Nahid to join efforts in implementing IPv6 live, and a pivotal point where we have in-house, full time, onsite talent working with our current local team. Our last meeting was yesterday 10am, and it was a first successful meeting. Additional access and follow up meetings are in process. We estimate we will have this work completed in 2-3 months pending the crisis. As of now, Nahid is rescheduled to fly out end of this month (August 2024). While writing up finalization of this report, Nahid was able to come back online with stable connectivity, and now has resumed remote work until he is flown out.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License 
