Over a year since Apple gave me an ultimatum on Windmill

Yesterday, I watched live a session by the House Judiciary Subcommittee on “Antitrust, Commercial, and Administrative Law | Online Platforms and Market Power, Part 6: Examining the Dominance of Amazon, Apple, Facebook, and Google.”

It so happened that yesterday, was the 1 year anniversary of the post Where is Windmill on the iPhone?.

Hearing the committee standing up for small business owners and individuals trying to support their family and employees, calling out corporations’ tactics on excercising control with a firm grip over people’s freedoms to operate on a free market while also hindering innovation brought back memories.

I got especially emotional hearing an audio clip of a book seller pleading to Jeff Bezos to restore their business operations so that they could earn their livelihood.

It’s fair to say that Apple got the least amount of attention and scrutiny by far.

Apple arguably got off the lightest of any of the companies in today’s hearing, if only by volume of questions: Tim Cook got just 35, compared to 59 for Bezos, 62 for Mark Zuckerberg, and 61 for Sundar Pichai

Casey Newton - Antitrust finally gets its hearing

With this enquiry as a landmark, I feel now is the time to share a few more details on what happened behind the scenes last summer as I was fighting to earn a living having spent countless months working on Windmill burning through the last of my savings. I want to also share some emails that capture my thoughts and emotions as I was going through the motions. I will also share, in full, the letter of appeal that I sent to the App Review Team.

Timeline

May 20, 2019: Windmill is approved on TestFlight for beta testing.
May 22, 2019: Windmill is “Waiting for Review” in the App Store.
June 12, 2019: I send a “Status Request”.
June 22, 2019: I email Phil Schiller bringing the case to his attention.
June 22, 2019: I get a call from the App Review Team where I am being given the reasons why Windmill on the iPhone will be rejected.
June 22, 2019: Windmill is “Rejected”.
June 26, 2019: I send an 8 pages “Letter of Appeal” to the App Review Team with the following:

  1. An overarching description that explains, what Windmill does, why I think it’s useful for developers, why it does not violate the App Store Guidelines and Apple Developer Program License Agreement, how Apple benefits by approving Windmill.
  2. The security considerations and implementation details that went into Windmill, for full transparency in an effort to make Apple feel comfortable about Windmill.
  3. In detail, why I believe Windmill complies with the App Store Review Guidelines and the Apple Developer Program License Agreement
  4. Final thoughts

July 3, 2019: “Based on the current material, the original rejection feedback remains valid.”
July 3, 2019: I send a message to the App Review Team asking for clarification on what grounds Windmill is rejected and for feedback to help me work towards making Windmill compliant.
July 9, 2019: I get a call from the App Review Team where I am being told that Windmill, “holistically” is not appropriate. This decision was made in collaboration with the App Review Board.
July 12, 2019: I email a developer tools evangelist at Apple asking for help to find a way forward with Windmill and for some some course correction to be done.

I never hear back.

Correspondance

Hello Phil,

My name is Markos and I am an independent Apple Developer living in Athens, Greece.

Apple’s developer relations and customer service has been overwhelmingly positive throughout the years, kudos for that.

In this rare case however, the App Review process seems to have fallen a victim of a scripted, no individual stepping forward, corner case where it just falls apart.

I will just give you the gist of it.

Windmill has been In Review for 21 work days, i.e. since the 23rd of May. I made a Status request 7 work days ago, i.e. since the 12th of June, It’s been about 12 hours since I have asked for some visibility on the status.

I wanted to bring this to your attention as this is now the cause of anxiety. I would very much appreciate you looking into this. I don’t expect a personal reply.

Thank you
.-
Markos

Seemingly, this email prompted the call I received on the very next (US) day.

I strenuously tried to understand Apple and listen to its concerns in an effort to meet its demands even though there was very little guidance in making Windmill compliant. At the same time, I vigorously made the case for Windmill’s approval. Alas, it eventually became clear to me that the call was a “courtesy” call. The decision had already been made and I was just being told how it is going to be.

I was confident this was a misunderstanding. That Windmill was not seen by Apple as a serious endevour given that I was just an indie developer. I was not ready to give up. I started drafting a letter of appeal with some feedback from fellow developers.

On 24 Jun 2019, at 12:29, Markos Charatzas markos@qnoid.com wrote:

Having had some time to think about the rejection, I now get the impression that Apple strongly believes Windmill on the iPhone to be no different to TestFlight which couldn’t be further from the truth.

Also, since Apple itself hasn’t really been much in support of the development process outside of Xcode, I worry that it may not believe in the value of Windmill. It does not help that Apple has a tendency of having a narrative that everything must fit into it.

2 days later, I sent a Windmill Letter of Appeal to the App Review Team. Windmill gets rejected again.

From:  Apple
Date: 09/07/2019, 19:13
2. - 5 Performance: Software Requirements
3. - 1.2 Business: Payments - Subscriptions
5. - 2.5 Legal: Intellectual Property - Apple Products Other - Other
Date: 03/07/2019, 07:02

Hello Markos,

Thank you for your detailed response and ongoing patience.

Based on the current material, the original rejection feedback remains valid.

However, if you believe your app is compliant with the App Store Review Guidelines, you are certainly well within your right to file an appeal to the App Review Board. As a member of the Apple Developer Program, you may exercise this option at any time.

Best regards,
[Redacted]
App Store Review

Date: 03/07/2019, 07:37
Hello [Redacted],

It is not clear to me whether the App Review team believes Windmill is still in violation of every App Store Review Guideline as well as the Apple Developer Program License Agreement. Can you please clarify whether Windmill is in compliance with some or all? Your feedback can help me work towards making Windmill compliant.

In regards to filing an appeal to the App Review Board. Given that the form to submit an appeal, only accepts plain text, how can I best send the letter in the PDF format?

Appreciated,
Markos

Date: 05/07/2019, 13:13
Hello [Redacted],

I would love to get on a call with you so that I can better understand how to proceed. Not sure what your schedule is like. I am generally available in the next few days and happy to talk during the weekend like last time.

By the way, I noticed that the number to contact you, [+1-XXX-XXX-XXXX], does not support FaceTime audio. It would make a positive difference to have that option available.

Regards,
Markos

From:  Apple
Date: 09/07/2019, 19:13
2. - 5 Performance: Software Requirements
3. - 1.2 Business: Payments - Subscriptions
5. - 2.5 Legal: Intellectual Property - Apple Products Other - Other

Hello Markos,

Thank you again for your time on the phone today, and further discussing the App Review Guidelines.

Whereas we are unable to provide you with specific development feedback for your app, we are dedicated to evaluating any revisions you would like to make in the form of a future resubmission.

Best regards,
[Redacted]
App Store Review

In the phone call I stated my intention to appeal. While I was told it was certainly within my right, I was given the impression that it wouldn’t make much of a difference. The decision was made having already consulted and factored in feedback from people outside the App Review Team.

It was on that final call, where it became pretty clear that Apple wasn’t willing to work with me on this. It was a call to tell me how this is going down. It is striking how differently Apple conducts itself on a call versus in writing; in public versus in private. The call sounded eerie familiar to the one made to Kapeli back in 2016 in regards to Dash. Dash and Apple: My Side of the Story.

“Has Apple ever retaliated against or disadvantaged a developer who went public about their frustration with the App Store?”

Going public

Having been given an ultimatum by Apple, I was encouraged by fellow developers to go public with the story. I was hesitant.

On 24 Jul 2019, at 08:07, Markos Charatzas markos@qnoid.com wrote:

  1. What is the end goal? Apple to reverse course and approve Windmill on the iPhone? Apple to change its stance, that holds privately, when it comes to how Command Line Tools can be used? Apple to have a public debate about developer tools on and about iOS?

  2. What is the case I am making and on what grounds? What’s possible to ship as an Apple Developer on Apple platforms, how the App Store is the only way to distribute iOS apps and the role of Apple as a gatekeeper? How that hinders innovation, competition and user choice?

These subjects have been debated for some time and right now are at the forefront even. How and why would they make a difference now?

Writing what Windmill does, how it’s helpful to developers, why it was rejected is only the setup. I don’t see that being sufficient to either get others’ support or for Apple to take this seriously.

Apple’s stance has caused me grief and a nihilistic view about the future of Windmill. At the same time, having Apple’s decision that Windmill holistically violates the Developer License Agreement hanging over my head is something I want to get off of me.

To be honest, I don’t feel comfortable, confident or keen on taking Apple, leading a movement or becoming a martyr in the process. I wish I could just go back to building Windmill and helping businesses get on board. I just feel that I can’t do that either.

Thank you for going through this with me.

On 24 Jul 2019, at 22:56, Markos Charatzas markos@qnoid.com wrote:

It’s possible that I’m overly emotional and choose to complicate things. I generally want to be well calculated and get things right. Even more so when I’m about to go on the public record with Apple when my developer license is on the line. It doesn’t help that have been beaten down by Apple twice already on this case.

This comes after more than 4 weeks of the App Review Team, the App Review Board and possibly Phil Schiller looking into Windmill on the iPhone. Also taking into account the 8 pages letter where I did say to Apple what the app does, how it works, and why the app would be helpful to other developers. I just can’t see how that alone would rally people. I might just be pessimistic.

After that letter, Apple decided to come after Windmill on the Mac saying I violate my developer license by building on top of the command line tools. This is more than simply an iPhone app being rejected.

The way I understand it, Apple does not want to set a precedent or open the door for others to exploit.

Keep in mind that Windmill on the Mac is the only 3rd party native macOS app that offers “continuous delivery”. The other one is Xcode bots by Apple itself. Any other service that uses the command line tools is on the web hence outside of Apple’s influence. Thus yet again being the only one directly affected by this. I believe you are using CircleCI so you know what I’m talking about.

I left London behind, a high paying job and have sacrificed at least 2 years of my life to build Windmill. I feel a lot of pressure and have a lot to lose on this so it doesn’t come easy.

It would help if you could tell me what you see in Windmill? What do you think is at stake? How does the decision by Apple to allow or sustain the rejection affect you as an Apple developer? Why do you think others would care? Why do you care? :) knowing that can help me build confidence.

On 26 Jul 2019, at 22:31, Markos Charatzas markos@qnoid.com wrote: Thanks!

So far I’ve had 2 more people giving me feedback.

One of them is [Redacted], who is definitely looking at this from a different, if not pessimistic angle. Which is good! Cause that means he does a great job covering the blind spots.

He made me think harder about what Apple is probably concerned about with Windmill. Creating a distribution mechanism which is available to the public, outside of the control of Apple and lacking the safety of the App Store. No matter how limited ad-hoc distribution may be.

Hard to debate about this as it brings the discussion back to the pros and cons of having the App Store as the only way to distribute apps…

Do you think there is a way to address this in the post without sounding like I’m not acknowledging the risk or sound insensitive to what Apple is concerned about?

I now think the “Final thoughts” section should be completely overhauled to touch on this issue and better communicate what I’m looking to get out of this by inviting a public debate/discussion.

I’m also thinking about removing both the “Timeline” and “Compliant” sections to keep the post shorter. I am not sure they add to anything. What do you think?

On 26 Jul 2019, at 23:12, Markos Charatzas markos@qnoid.com wrote:
But in some way, it does. Windmill today is a public offering/service. It doesn’t matter what Windmill is designed to be used for but how it is implemented and how it could be used for.

It is precisely why Apple had to say that the Command Line Tools Package is only meant to be used by developers alone.

I feel like I’ve completely misread the situation with Apple up until now.

Say, if an organisation had built Windmill for internal use. The server used for distribution would be under their control and probably in their internal network.

Windmill on the iPhone would probably be distributed using an Enterprise Certificate, internally to the employees of the organisation or using ad-hoc distribution. It doesn’t matter, it’s internal.

It completely changes how organisations would use Windmill. In this case, I am not even sure a public post is useful.

What an epiphany.

Are you following my train of thought at all? :)

I have removed the receipient in the emails above to focus on my thoughts, considerations and reasoning instead. For the final exchange though, I would like to include it as I believe it makes an important point about the App Store as the only means of distributing iOS apps.

I think I’m following you. The topic is way outside my normal areas of expertise, though.

So, to be clear — could Windmill be used to distribute apps to the public, circumventing TestFlight and the App Store?

On 27 Jul 2019, at 14:18, Markos Charatzas markos@qnoid.com wrote:

You can’t use Windmill to distribute apps to the public, circumventing TestFlight and the App Store any more than you can without using Windmill.

It’s nuanced.

The rest is history.

What is this website?

This is a personal website, at the outskirts of the web, away from social media and publishing platforms. This website surfaces social, racial, economic traits and explores human relationships. It highlights the conditions that contribute to one's personal success or downfall. It shares stories that act as a reminder that life is messy, complex, nuanced, diverse. It aims to bring the world closer together. It reaches out to those that feel lost, lonely, inadequate and outcasts. I am with you.

for historic purposes, 20th to 21st century