Pro Charger Connection issue

[Joedirt]

Had my charger installed in may but have had internet connection issues since the beginning.
it randomly disconnects and doesn’t update my insights when it reconnects.
this is a company truck and this is how I get paid for my home electric usage.


internet isn’t an issue as the router is less than 11’ away and signal is strong.

see the gaps in the attached pics. There’s a lot of gaps not tracked and it’s impossible to get an accurate picture of usage.
I would think the charger would store and update when connected but this is not the case.

thinking about just installing an external meter at the j box below the charger that I can manually record usage.

Sponsored

 

[TaxmanHog]

When you look at these screens do you see what I have?
On occasion it will fail to connect to cloud database, when that happens I power cycle the FCSP, wait 5 to 30 minutes and communications usually are restored and some times missed record will post, though over the last 11 months of operation I've had several lost sessions.

 

[TaxmanHog]

BTW: a new firmware update has been released today, hopefully it will improve reliability of connections and data reporting.

 

[Joedirt]

Yes sir. Except when it’s not. It seems to not be able to reconnect on its own. I read another post that suggested shutting off the breaker and resetting it which seems to help it reconnect. But that’s only when I notice it has disconnected.
hopefully this gets resolved soon but if not I’m going to have to add a meter in line so I can do a better job of tracking useage.

 

[TaxmanHog]

hopefully this gets resolved soon but if not I’m going to have to add a meter in line so I can do a better job of tracking usage.

I installed this on the 100 amp 240v feed to my FCSP, works perfectly logs all my sessions never lost a charging event.

 

[Joedirt]

I was looking at this today. Thank you for replying. I like this because I can also pinpoint power usage elsewhere. We’re on solar but it only produces during the day, so the nights are all APS. We.re not their customer anymore so they hit us pretty hard when we have to pull from the power grid. We had an issue with our solar when the charger went in and our June and July electric bills were $889 and $857 respectively. Nailing down the charger usage is paramount at this point.

 

[djwildstar]

In my experience, the Ford Charge Station Pro and FordPass power reporting is currently broken (meaning that it is of little or no possible value to anyone). The two issues that I see are:

• The FCSP reports the entire charge session as power used in the hour that the truck was unplugged. So (for example), if you charged at 19.2kW from 11pm yesterday to about 12:30am today for 30kWh total, instead of reporting 19.2kW in the 11pm-midnight hour and 10.8kWh in the midnight-1am hour, it reports 30kWh in the 7am-8am hour when you unplugged the truck. How is this useful to anyone for anything?.
• FordPass can use the charging data from the FCSP to compute cost to charge the truck. However, FordPass only supports a single rate -- so the cost calculation is only correct for users who have a flat-rate plan. People who have time-of-use or EV-specific plans have no ability to put in different rates at different times -- and even if they did, the charge session reporting data from the FCSP means that it would be calculated in the wrong time period regardless.

Hopefully the pending update will address some of these issues, but actually I'd prefer it if the update fixes the ongoing problems with the home integration system and intelligent backup power. It also seems likely that updates to FordPass could address some of these issues, such as the lack of support for different rates at different times during the day.

 

[TaxmanHog]

In my experience, the Ford Charge Station Pro and FordPass power reporting is currently broken (meaning that it is of little or no possible value to anyone). The two issues that I see are:

• The FCSP reports the entire charge session as power used in the hour that the truck was unplugged. So (for example), if you charged at 19.2kW from 11pm yesterday to about 12:30am today for 30kWh total, instead of reporting 19.2kW in the 11pm-midnight hour and 10.8kWh in the midnight-1am hour, it reports 30kWh in the 7am-8am hour when you unplugged the truck. How is this useful to anyone for anything?.
• FordPass can use the charging data from the FCSP to compute cost to charge the truck. However, FordPass only supports a single rate -- so the cost calculation is only correct for users who have a flat-rate plan. People who have time-of-use or EV-specific plans have no ability to put in different rates at different times -- and even if they did, the charge session reporting data from the FCSP means that it would be calculated in the wrong time period regardless.

Hopefully the pending update will address some of these issues, but actually I'd prefer it if the update fixes the ongoing problems with the home integration system and intelligent backup power. It also seems likely that updates to FordPass could address some of these issues, such as the lack of support for different rates at different times during the day.

Accurate observations, and why my neighbors have seen me running out to the garage at odd hours of the evening to unplug and capture the cumulative charge value. Fruitless efforts for all the above reasons. IMHO, they won't improve the logging methodology until the data processing bottleneck / capacity is improved, I think that's why we see it summarized on disconnect instead of incremental by the second as other systems are capable of logging.

 

[djwildstar]

IMHO, they won't improve the logging methodology until the data processing bottleneck / capacity is improved, I think that's why we see it summarized on disconnect instead of incremental by the second as other systems are capable of logging.

In my opinion, sending data when an "event" (like a disconnect) happens isn't so much the issue. For an Internet-connected embedded device, that isn't a bad design pattern -- it reduces workload and memory consumption on the controller, since the network stack can be unloaded or disconnected most of the time. On the server side, processing a small table of numbers (say, usage in 5-minute blocks going back for the past month, about 32kbytes of data) sent in response to an event is practically as fast as processing a single number. Most of the time is spent in overhead tasks (setting up connections, authentication, etc.) rather than actually sending and processing data.

Coming from a software background (including embedded software) I suspect that the problem is fundamentally one of bad software development process. I've seen it before in organizations that don't have strong software development culture, lack good communication between the various teams, have outsourced portions of the work, or are under heavy time pressure. I suspect that all of these applied at Ford during the development of the Charge Station Pro, Lightning, and associated systems.

I'm guessing the scenario went something like this: Management wanted the Charge Station Pro to report usage via a web-connected service so that Ford can provide charging data to owners, and so that Ford can collect charging and usage data for product research. The requirement goes through several more people without a lot of thought until it lands in the inbox of a junior developer or offshore contractor, who reads something like "The embedded system will send date/time and usage data to this web API when one of these events occurs". Like most developers, this one wants to clear tickets as promptly as possible ... so thinks "Events, no problem, we get an interrupt when one of these events happens. A web API is easy-peasy, so what data do we send? Usage data? Do they mean kWh used? Okay, I'll accumulate total kWh sent in a variable, that's sorted. Now, date/time? What date/time should I send? Event date/time is the only thing that makes sense. And I guess I'll reset the usage variable back to zero once I get a confirmation from the API. Okay, done!".

The critical bit that was missing was the intent was to collect usage over time in discrete buckets. Every second would be awesome, but every 5 minutes would be fine; and even every half-hour or hour would be okay.

The rest of the ecosystem gets built out based on the single-number misunderstanding -- a web API to receive the data, some code to turn it into a report. They test it out and it works! Web sites and mobile apps updated to support it! Tickets closed! Eventually demo time comes and ... the product owner isn't happy: Why would anyone think this is what they wanted? Hasn't anyone seen one of the competing products usage reports? But at this point, they've got a hard deadline to ship a product -- so ship it as version 1, and address it with a software update when we have time to fix it.

But now what started as a small problem (the system owners failed to communicate to the developers exactly what data to collect) is now a bigger problem: there's a whole ecosystem consisting of an installed base of charge stations, a web services API, processing code and back-end storage, a web site, and mobile apps for both Android and iOS that has ALL been built around the misunderstood requirement. To fix it, functionality has to be added to all of these pieces to handle both the corrected v2 data format, without breaking the old v1 data flow -- because we can't guarantee that everything will get updated same time (or even will get updated at all), so we have to support both formats going forward. This is actually a harder problem than the initial implementation, and can require a good bit of testing to ensure that the new features don't break the old ones.

Or as John Wooden used to say "If you don't have time to do it right, when will you have time to do it over?"

 

[TaxmanHog]

Or as John Wooden used to say "If you don't have time to do it right, when will you have time to do it over?"

Poorly defined customer requirements for end users like us by FORD, the root of this issue as you perfectly described.

My closing years at IRS were spent working directly with a team of DEV's who designed and built a special enforcement processing system, we spent hours working on and refining the requirements of the original design and ongoing improvements, they were the technical folks working with me as the "customer" using the system to do case processing work. Upon my retirement, many wish list items remained on the task queue forever as they had insufficient funding to barely keep operations and current tax law modifications, so what I'm saying is economics and operations play into what goes into production.

 

[RedLightning86]

In my opinion, sending data when an "event" (like a disconnect) happens isn't so much the issue. For an Internet-connected embedded device, that isn't a bad design pattern -- it reduces workload and memory consumption on the controller, since the network stack can be unloaded or disconnected most of the time. On the server side, processing a small table of numbers (say, usage in 5-minute blocks going back for the past month, about 32kbytes of data) sent in response to an event is practically as fast as processing a single number. Most of the time is spent in overhead tasks (setting up connections, authentication, etc.) rather than actually sending and processing data.

Coming from a software background (including embedded software) I suspect that the problem is fundamentally one of bad software development process. I've seen it before in organizations that don't have strong software development culture, lack good communication between the various teams, have outsourced portions of the work, or are under heavy time pressure. I suspect that all of these applied at Ford during the development of the Charge Station Pro, Lightning, and associated systems.

I'm guessing the scenario went something like this: Management wanted the Charge Station Pro to report usage via a web-connected service so that Ford can provide charging data to owners, and so that Ford can collect charging and usage data for product research. The requirement goes through several more people without a lot of thought until it lands in the inbox of a junior developer or offshore contractor, who reads something like "The embedded system will send date/time and usage data to this web API when one of these events occurs". Like most developers, this one wants to clear tickets as promptly as possible ... so thinks "Events, no problem, we get an interrupt when one of these events happens. A web API is easy-peasy, so what data do we send? Usage data? Do they mean kWh used? Okay, I'll accumulate total kWh sent in a variable, that's sorted. Now, date/time? What date/time should I send? Event date/time is the only thing that makes sense. And I guess I'll reset the usage variable back to zero once I get a confirmation from the API. Okay, done!"
.....

Left one thing out - the day after the Lightning launched, they laid off something like a hundred software engineers. So the guy or two that are left, is always way behind the stack of demands in their inbasket. How much you want to bet that Tesla and Rivian and Arcimoto snapped those software engineers up? And the Big 3 whine about the union strike!

 

[djwildstar]

Do you have a source for the layoff of software engineers related to the Lightning launch?

Ford has had layoffs in 2022 and 2023 to try to cut costs. In 2022, there were about 2000 Ford employees and 1000 contractors laid off. This wasn't unusual for the industry last year: Rivian laid off 6% of their workforce around the same time, while Tesla laid off a chunk of the Autopilot team.

Ford has announced further layoffs for 2023. There hasn't been a lot of detail on what kind of staff are being let go. There has been a suggestion that the layoffs are related to the 2022 restricting and transition to EVs, and have hit Ford Blue (the ICE division) hardest.

Overall it isn't unusual to assemble (or hire) a software development team to build a new product, and then reassign (or lay off) the majority of the team when the project ships, leaving just a few to support and maintain the product.

Sponsored