GPT-4 개발자 라이브스트림 (OpenAI) :: ChatGPT 정리

> English Summary

> English Full Text

GPT-4 Developer Livestream
https://www.youtube.com/watch?v=outcGtbnMuQ 

 

 

[ 요약 ]

GPT-4 개발자 데모 라이브 스트림이 진행됩니다. 회사 설립 이후부터 OpenAI는 이 기술을 만들고 있었으나, 최근 2년간은 GPT-4를 출시하기 위해 모델을 훈련시키고, 위험성을 파악하고, 파트너와 함께 실제 시나리오에서 테스트하며, 모델을 최적화시키는 작업에 집중해왔습니다. 이번 데모에서는 GPT-4의 가능성과 한계, 여전히 작업 중인 부분을 보여주고, 좋은 도구와 파트너로서 GPT-4를 최대한 활용하는 방법을 소개할 예정입니다. 데모 스트림에 참여하고 싶다면 openAI의 디스코드에 가입하여 의견을 제시해주세요. 첫 번째로 GPT-4가 3.5에서는 할 수 없었던 작업을 소개할 예정이며, 훈련 도중 수행하는 작업들을 지속적으로 수행해나갈 것입니다.

GPT-4는 OpenAI에서 개발 중인 AI 모델로, 이전 모델보다 더 많은 작업을 수행할 수 있습니다. GPT-4가 수행할 수 있는 작업 중 하나인 챗봇이 있습니다. GPT-4의 API를 사용하여 사용자 메시지를 입력하면, 모델이 적절한 응답을 반환합니다. 이를 통해 챗봇 대화가 더 구조화되고, 모델이 더욱 신뢰성 높은 작동이 가능해집니다. 이러한 기술을 활용하여 더욱 발전된 AI 모델을 만들어 나갈 것입니다.

GPT-4는 글의 특정 단어로 시작하는 요약문을 작성할 수 있는 능력도 갖추고 있습니다.

GPT-4를 활용하여 기존 콘텐츠를 창조적으로 활용하고, AI 프로그래밍 어시스턴트를 만들 수 있습니다. OpenAI에서는 GPT-4의 성능을 평가하기 위한 오픈소스 평가 프레임워크를 제공하고 있으며, 이를 통해 모델의 한계를 파악하고 개선할 수 있습니다.

GPT-4 모델이 문서를 활용하여 새로운 내용을 합성하고 Discord API와 Jupyter Notebook에서 발생할 수 있는 오류를 처리합니다. 모델이 미리 학습한 정보를 새로운 방식으로 활용하여 새로운 콘텐츠를 생성하는 능력이 있으며, Discord API와 Jupyter Notebook에서 발생할 수 있는 오류를 처리하는 데도 유용합니다. 사용자는 모델이 생성한 코드를 항상 검토하고, 신뢰할 수 없는 코드를 실행하지 않아야 합니다. 또한, Discord API와 Jupyter Notebook의 버전이 모델이 학습한 버전과 다를 경우, 모델이 생성한 코드는 작동하지 않을 수 있으므로 주의해야 합니다. 이러한 오류를 처리하기 위해 모델에 오류 메시지를 입력하면 모델이 해당 오류를 수정하고 올바른 코드를 생성할 수 있습니다. 또한, 모델이 현재 환경에서 실행되는지 확인하기 위해 Jupyter Notebook을 사용할 때는 적절한 라이브러리를 설치하고, Discord API의 버전 문제를 해결하기 위해 모델이 제안하는 방법을 따르는 것이 좋습니다.

GPT-4 모델은 이미지를 인식할 수 있습니다. 예를 들어, Discord API를 통해 이미지와 텍스트를 혼합하여 입력할 수 있습니다. 모델이 이미지를 인식하기 위해서는 이미지에 대한 설명을 자세히 제공해야 하며, 모델은 제공된 정보를 기반으로 이미지를 분석합니다.
Discord에서는 메시지 내용을 인식하기 위해 intent 필드가 필요한데, 이를 활용하여 모델이 메시지를 인식할 수 있습니다. 단, 이 기능은 모델이 인식할 수 있는 범위를 벗어날 수 있으므로, 테스트를 통해 최적화할 필요가 있습니다. 

반응형

[ 한글 전체 ]

GPT-4 개발자 데모 라이브 스트림.
솔직히 이런 날이 왔다는 사실이 믿기지 않습니다.
OpenAI는 이 기술을 개발해 왔습니다.
이 기술을 개발해왔습니다,
하지만 지난 2년 동안
GPT-4를 제공하는 데 집중해 왔습니다.
이는 전체 트레이닝 스택을 재구축하는 것에서 시작되었습니다,
실제로 모델을 훈련시키고
그다음에는 모델이 어떤 기능을 하는지
그 능력을 파악하고
파트너와 협력하여
실제 시나리오에서 테스트하기 위해,
실제로 행동을 조정하고
모델을 최적화하고
사용할 수 있도록 제공하는 것입니다.
오늘은 GPT-4를 빛나게 하는 방법을 조금 보여드리고자 합니다,
어떻게 하면 GPT-4를 최대한 활용할 수 있는지,
약점은 무엇인지,
아직 해결해야 할 부분,
그리고 좋은 도구, 좋은 파트너로 사용하는 방법도 알려드리겠습니다.
따라서 스트림에 참여하고 싶으시다면,
저희 디스코드에 가시면
discord.gg 슬래시 오픈AI입니다,
거기에 댓글을 남기시면
몇 가지 시청자 제안을 받겠습니다.
가장 먼저 보여드리고 싶은 것은 첫 번째 작업입니다.
3.5에서는 할 수 없었던 작업을 GPT-4에서 할 수 있다는 것입니다.
이에 대해 생각하는 방법은 훈련 내내 내내 있습니다,
이 모든 작업을 지속적으로 수행한다는 것입니다.
새벽 2시에 호출기가 울리고,
모델을 고치면서 항상 궁금해지죠,
잘 될까?
이 모든 노력이 실제로 효과가 있을까요?
그래서 우리 모두는 각자 정말 좋아하는 애완동물 과제가 있었어요,
그리고 우리 모두는 개별적으로 그것을 보려고 노력할 것입니다,
이 모델이 지금 할 수 있을까요?
첫 번째는 4인용은 성공했지만
하지만 3.5에서는 성공하지 못했습니다.
오늘 블로그 게시물의 상단을 복사해서
플레이그라운드에 붙여넣겠습니다.
이제 2주 전에 출시된 새로운 채팅 완성 기능인 Playground
입니다.
먼저 GPT-3.5를 통해 보여드리겠습니다,
4와 동일한 API, 동일한 플레이그라운드가 있습니다.
작동 방식은 시스템 메시지가 있습니다.
시스템 메시지를 통해 모델에 무엇을 해야 하는지 설명할 수 있습니다,
그리고 이 모델들을 매우 조정 가능하게 만들었죠,
원하는 모든 지시를 내릴 수 있습니다,
원하는 모든 지시를 내릴 수 있습니다.
그리고 모델은 이를 잘 따를 것입니다,
앞으로는 점점 더 강력하게
모델을 매우 안정적으로 조종할 수 있게 됩니다.
그런 다음 사용자가 원하는 것을 붙여넣을 수 있습니다,
모델이 어시스턴트로서 메시지를 반환합니다.
이렇게 생각하면 원시 텍스트를 입력하는 방식에서
원시 텍스트를 입력하고 원시 텍스트를 출력하는 방식에서 벗어나고 있습니다,
대화의 다른 부분이 어디에서 왔는지 알 수 없는 것에서 벗어나
훨씬 더 구조화된 형식으로 전환하여
모델에게 알 수 있는 기회를 제공합니다,
음, 이것은 사용자가 저에게 무언가를 요청하는 것입니다.
개발자는 참석하지 않았습니다,
여기서 개발자의 말을 들어야겠군요.
자, 이제 실제로 보여드릴 시간입니다.
실제로 보여드릴 시간입니다.
여러분도 잘 아시겠지만, 이걸 요약해 보세요,
그리고 기사를 문장으로 표현하고, 좀 더 구체화해서
좀 더 구체적으로, 하지만 모든 단어가 g로 시작하는 곳입니다.
이제 3.5가 어떻게 작동하는지 보겠습니다.
네, 시도조차 하지 않고 그냥 작업을 포기했습니다.
3.5가 이런 종류의 작업을 시도할 때
이런 종류의 작업을 시도하는 것은 매우 일반적입니다.
만약 이 기사가 아주 삐뚤어진 기사나 그런 거라면
성공할 수도 있겠지만 대부분의 경우 포기합니다,
3.5는 그냥 포기합니다.
하지만 정확히 동일한 프롬프트, 정확히 동일한 시스템에서
메시지가 나타납니다.
AI를 포함할지 말지 경계가 모호한 상황입니다.
하지만 AI가 포함되지 않는다고 가정해 보겠습니다.
공평하게도 모델은 제 피드백을 기꺼이 받아들입니다.
이제 G에게만 좋은 것이 아니라는 것을 확실히 하기 위해,
이 문제를 청중에게 넘기고 싶습니다.
다음에 시도할 편지에 대한 제안을 받겠습니다.
그동안 사회자가 행운의 주인공을 뽑는 동안
행운의 편지를 고르는 동안 제가 A로 시도해볼게요.
하지만 이 경우에는 GPT-4가 괜찮다고 말씀드리겠습니다.
왜 안 될까요?
또한 요약도 꽤 잘 되어 있죠.
그럼 이제 디스코드로 넘어가볼게요. 
좋아요, 와우.
사람들이 조금 야심 차게 굴고 있다면,
전 정말 모델을 시험해보고 싶어요.
Q를 시도해볼 건데, 잠시만 생각해보면
잠시만 생각해보면 청중들이 정말 생각해봤으면 합니다,
이 기사를 어떻게 요약할까요?
어떻게 요약할까요?
쉽지 않죠.
꽤 괜찮네요.
꽤 좋네요.
기존 기사를 요약하는 방법을 보여드렸습니다.
이제 서로 다른 기사 간에 아이디어를 유연하게 결합하는 방법을
유연하게 결합하는 방법을 보여드리겠습니다.
그래서 어제 해커 뉴스에 실렸던
어제 해커 뉴스에 있던 기사를 가져와서 복사해서 붙여넣고
같은 대화를 나누면서 방금 전에 했던 일의 모든 맥락이
우리가 방금 한 일의 모든 맥락을 가지고 있습니다.
이 기사와 GPT-4의 공통 주제를 하나 찾아보겠습니다.
하나의 공통 주제를 찾아보겠습니다.
이 글은 파이썬 웹 앱 개발 프레임워크인 Pinecone에 대한 글입니다.
앱 개발 프레임워크인 Pinecone에 대한 기사입니다.
기술을 더 접근하기 쉽고 사용자 친화적으로 만드는 것입니다.
충분히 통찰력이 없었다고 생각하신다면,
언제든지 피드백을 주시면 됩니다,
충분히 인사이트가 없었다고요.
제발요.
아니요, 그냥 그대로 두겠습니다.
모델이 결정하도록 맡기세요.
강력한 기술과 실제 애플리케이션 사이의 간극을 좁히는 것은
실용적인 애플리케이션 사이의 간극을 좁히는 것도 나쁘지 않은 것 같습니다.
물론 유연한 언어 이해력을 이용해 원하는 다른 종류의 작업을
유연한 언어 이해와
및 합성.
예를 들어, 이제 GPT-4 블로그 게시물을
운율 있는 시로 바꿔주세요.
모두를 위한 오픈 소스인 IE 밸브 열기를 포착했습니다,
전화 응답을 안내하는 데 도움이 됩니다,
이 모델에 기여하고 싶으시다면
E-Val을 보내주세요.
저희는 오픈 소스 평가 프레임워크
오픈 소스 평가 프레임워크가 있습니다.
이해하고 다음 단계로 나아가는 데 도움이 되는 오픈 소스 평가 프레임워크가 있습니다.
여기까지입니다.
이것은 GPT-4를 사용하여 기존 콘텐츠를 소비하는 것입니다.
를 사용하여 기존 콘텐츠를 소비하는 것입니다.
이제 GPT-4로 빌드하는 방법을 보여드리겠습니다.
파트너와 함께 제작하는 것이 어떤 것인지 보여드리겠습니다.
그래서 우리가 할 일은
실제로 Discord 봇을 만들어 보겠습니다.
실시간으로 빌드하고 그 과정을 보여드리고 디버깅하는 모습을 보여드리겠습니다,
모델이 무엇을 할 수 있는지, 한계는 어디인지,
그리고 새로운 차원을 달성하기 위해 어떻게 작업해야 하는지 보여드리겠습니다.
그래서 제가 가장 먼저 할 일은 모델에 다음과 같이 말하는 것입니다.
이번에는 AI 프로그래밍 어시스턴트가 될 것이라고 말입니다.
이 모델의 임무는 먼저 의사 코드를 작성하고
그리고 나서 실제로 코드를 작성하는 것입니다.
이 접근 방식은 매우 유용합니다.
모델이 문제를 더 작은 조각으로 나누도록 하는 데 매우 유용합니다.
이렇게 하면 모델에게
문제에 대한 매우 어려운 해결책을 한 번에
요구하지 않아도 됩니다. 
또한 해석이 매우 쉽습니다,
모델이 무슨 생각을 했는지 정확히 알 수 있기 때문입니다,
원하는 경우 수정 사항을 제공할 수도 있습니다.
그래서 여기에 우리가 물어볼 프롬프트가 있습니다.
이것은 3.5가 완전히 질식할 수 있는 종류의 것입니다.
질식할 것 같은 문제입니다.
그래서 저희는 GPT-4 API를 사용하여
이미지와 텍스트를 읽는 디스코드 봇을 요청하겠습니다.
여기서 한 가지 문제가 있는데, 바로 이 모델의 트레이닝입니다.
컷오프가 2021년이기 때문에 새로운 채팅 완료 형식인
완료 형식을 보지 못했다는 것입니다.
그래서 말 그대로 2주 전의 블로그 게시물로 이동했습니다,
응답 형식을 포함하여 블로그 게시물에서 복사하여 붙여 넣었습니다.
새로운 이미지 확장을 보지 못했습니다.
그래서 저는 이미지를 포함시키는 방법에 대해 아주 간단하게
이미지를 포함시키는 방법에 대해 아주 간략하게 설명했습니다.
이제 모델은 실제로 그 문서를 활용할 수 있습니다.
외우지 않아도 되고, 알지 않아도 됩니다.
그리고 일반적으로 이러한 모델은 학습된 정보를 매우 잘 활용합니다.
새로운 방식으로 학습한 정보를 사용하고
새로운 콘텐츠를 합성하는 데 매우 능숙합니다.
그리고 바로 여기서 보실 수 있습니다.
완전히 새로운 봇을 작성했습니다.
이제 실제로 이 봇이 실제로 작동하는지
실제로 작동하는지 살펴봅시다.
따라서 항상 코드를 살펴보고
코드를 살펴봐야 합니다.
사람이나 AI의 신뢰할 수 없는 코드를 실행하지 마세요.
그리고 한 가지 주의해야 할 점은 Discord API가
가 시간이 지남에 따라 많이 변경되었다는 점입니다.
특히, 한 가지 주목할 점은
이 모델이 학습된 이후 많은 변화가 있었습니다.
바로 '사용해보십시오.
사실, 예, 강렬한 키워드가 빠져 있습니다.
이것은 2020년에 나온 기능입니다.
따라서 모델은 이 키워드가 존재한다는 것을 알고 있습니다,
하지만 어떤 버전의 Discord API를 사용하고 있는지
알지 못합니다.
그럼 운이 없는 건가요?
그렇지는 않습니다.
오류 메시지를 정확히 모델에 붙여넣기만 하면 됩니다.
메시지를 모델에 붙여넣기만 하면 됩니다,
이 오류는 코드를 실행할 때 발생합니다.
고쳐주실 수 있나요?
그냥 실행하면 됩니다.
그러면 모델이 '네'라고 대답합니다.
웁스, 격렬한 논쟁이 벌어졌네요.
여기 올바른 코드가 있습니다.
이제 코드가 무엇을 하는지 다시 한 번 확인해 보겠습니다.
코드가 무엇을 하는지 다시 한 번 확인해 봅시다.
이제 두 번째 문제가 발생할 수 있습니다.
내가 어떤 환경에서 실행 중인지 모른다는 것입니다.
그리고 눈치채셨다면, 다음과 같이
이해할 수 없는 오류 메시지가 표시됩니다.
Jupyter Notebook을 많이 사용해 보지 않으셨다면 아마도
이게 무슨 뜻인지 전혀 모르실 겁니다.
하지만 다행히도 다시 한 번 말씀드리자면, 이 오류 메시지는
모델에게 "안녕하세요, 저는 Jupyter를 사용하고 있습니다.
저는 Jupyter를 사용하고 있고, 이걸 작동하게 만들고 싶습니다.
고쳐주세요.
그리고 구체적인 문제는 이미 실행 중인 이벤트
루프가 실행 중이므로 이 네스트 async.io
라이브러리를 사용해야 합니다.
nest async.io.apply를 호출해야 합니다.
모델은 이 모든 것을 알고 있습니다.
이 모든 부분을 봇에 올바르게 인스턴스화합니다.
심지어 Jupyter에서 실행 중이라는 사실도 알려줍니다.
패키지를 아직 설치하지 않은 경우, 이 뱅, 핍 인스톨을 수행하여
패키지를 설치할 수 있습니다.
매우 도움이 되었습니다.
이제 실행해보니 문제가 발생한 것 같습니다.
가장 먼저 할 일은 Discord로 이동하여
디스코드 자체의 스크린샷을 붙여넣겠습니다.
GPT-4는 단순한 언어 모델이 아니라는 점을 기억하세요.
비전 모델이기도 합니다.
실제로 이미지와 텍스트가 산재되어 있는
이미지와 텍스트를 임의로 섞어 넣을 수 있습니다.
문서처럼요.
이제 이미지 기능이 미리 보기에 있습니다.
그래서 이 기능을 살짝 엿볼 수 있습니다.
아직 공개적으로 사용할 수 없습니다.
이 기능은 우리가 한 파트너와 함께 작업하고 있는 것입니다.
Be My Eyes라는 파트너와 협력하고 있습니다.
프라임 타임에 맞춰 준비 중입니다.
하지만 궁금한 점이 있으시면 무엇이든 물어보세요. 
예를 들어, GPT-4, 안녕하세요.
이 이미지를 자세하게 설명할 수 있나요?
좋아요, 우선 어떻게 할 것인지 생각해보세요.
직접 해보세요.
여러분이 붙잡을 수 있는 것은 여러 가지가 있겠죠.
시스템의 다양한 부분을 설명할 수 있습니다.
그리고 실제 코드로 이동하면 다음과 같은 것을 확인할 수 있습니다.
네, 실제로 메시지를 받았고, 적절한 형식의 요청을
API에 적합한 요청 형식을 지정했습니다.
이제 우리가 해야 할 일 중 하나는
시스템을 더 빠르게 만들어야 하기 때문입니다.
이것이 우리가 최적화하기 위해 노력하고 있는 것 중 하나입니다.
그 동안 시청자 여러분께 드리고 싶은 말씀이 있습니다.
시청자 여러분, 다음에는 시청자 요청을 받겠습니다.
여러분이 원하시는 이미지나 하고 싶은 작업이 있다면
디스코드에 제출해 주세요.
진행자가 한 가지를 골라 진행할 것입니다.
그러면 디스코드에서 "오, 응답이 있는 것 같네요.
응답이 있네요.
완벽하네요.
Discord 애플리케이션 인터페이스의 스크린샷입니다.
꽤 좋네요.
설명하지도 않았습니다.
디스코드라는 것을 알고 있습니다.
아마 어딘가에 디스코드라고 쓰여 있을 거예요.
이전 경험을 통해 알고 있을 겁니다.
GPT-4라고 표시된 서버 아이콘은 인터페이스를 아주 자세하게 설명합니다.
아주 자세하게 설명되어 있습니다.
대기열을 해야 한다고 말하는 모든 사람들에 대해 이야기합니다.
대기열을 하라고 합니다.
매우 친절한 청중입니다.
그리고 많은 알림 메시지와 채널에 있는 사용자에 대해
채널에 있는 사용자에 대해서도 설명합니다.
이렇게 설명합니다.
이제 꽤 잘 이해가 되셨을 겁니다.
이제 다음 장면을 보시면 우선 게시물이 있지만
게시물이 있지만 모델이 실제로 메시지를 보지 못했습니다.
그렇다면 이것은 모델에 문제가 있는 건가요?
모델의 실패일까요?
글쎄요, 우리가 살펴볼 수 있습니다.
보시다시피, 여기서 콘텐츠는 빈 문자열입니다.
빈 메시지 내용을 받았습니다.
그 이유는 우리가 AI에 대해
속임수를 썼기 때문입니다.
따라서 Discord 문서로 이동하여 스크롤을 내려가면
스크롤하여 메시지 내용 의도까지 내려가 보세요.
2022년 9월부터 이 항목이 필수 항목으로 추가된 것을 볼 수 있습니다.
필수 필드입니다.
따라서 명시적으로 나를 태그하지 않은 메시지를 받으려면
메시지를 받으려면 이제 코드에 이 새로운 인텐트를 포함해야 합니다.
코드에 포함시켜야 합니다.
인텐트는 시간이 지남에 따라 많이 변경되었다는 점을 기억하세요.
이것은 모델이 알 수 있는 것보다 훨씬 새로운 것입니다.
알 수 있는 것보다 훨씬 새롭습니다.
그래서 운이 나빴을 수도 있습니다.
직접 디버깅해야 합니다.
하지만 다시 한 번 GPD4의 언어 이해 기능을 사용하여
기능을 사용하여 이 문제를 해결할 수 있습니다.
명심하세요, 이 문서는 약 10,000개, 15,000개 정도의
10,000단어, 15,000단어 정도 되는 문서입니다.
형식이 잘 갖춰져 있지 않습니다.
이것은 말 그대로 복사 붙여넣기 명령입니다.
이것이 문서 중간에 정의되어 있는
문서 중간에 있는 메시지 내용입니다.
이제 필수입니다.
하지만 할 수 있는지 봅시다.
빈 메시지를 받고 있습니다.
내용.
왜 이런 일이 발생할 수 있나요?
어떻게 고칠 수 있나요?
GPD4의 새로운 기능 중 하나는 컨텍스트 길이입니다.
32,000 토큰은 현재 저희가 지원하는 상한선입니다.
상한선입니다.
그리고 이 모델은 긴 문서를 유연하게 사용할 수 있습니다.
아직 최적화 중입니다.
따라서 사용해 보시기를 권장하지만, 아직은 확장하지 않는 것이
아직은 확장하지 않는 것이 좋습니다.
아직은 확장하지 않는 것이 좋습니다.
따라서 긴 문맥에 정말 관심이 있으시다면
알려주세요.
어떤 종류의 애플리케이션이 이 기능을 활용할 수 있는지 보고 싶습니다.
하지만 보시면 알겠지만, 메시지 콘텐츠 의도가
가 비활성화되지 않았습니다.
따라서 모델에 코드를 작성하도록 요청하거나
코드를 작성하도록 요청할 수도 있고, 아니면 실제로 옛날 방식으로
방법.
어느 쪽이든 괜찮습니다.
저는 이것이 증강 도구라고 생각합니다.
더 생산적이죠.
하지만 여전히 중요한 것은 여러분이 운전석에 앉아서,
관리자로서 무슨 일이 일어나고 있는지 알고 있어야 합니다.
이제 다시 한 번 연결되었습니다.
보리스, 메시지를 다시 재생할까요?
다시 한 번 봇이 명시적으로 메시지를 보내지 않았지만
봇에 명시적으로 태그가 지정되지 않았음에도 불구하고요.
꽤 괜찮은 설명인 것 같네요.
흥미롭네요.
사실 이것은 흥미로운 이미지입니다.
돌리가 생성한 이미지인 것 같습니다.
이것도 실제로 시도해 봅시다.
이 이미지의 어떤 점이 재미있나요?
아, 이미 제출된 이미지입니다.
다시 한 번, 올바른 API 호출을 하고 있는지 확인할 수 있습니다.
호출하고 있는지 확인할 수 있습니다.
다람쥐는 일반적으로 견과류를 먹습니다.
카메라를 사용하거나 사람처럼 행동할 것이라고는 생각하지 않습니다.
그래서 저 이미지가 왜 재미있는지
이미지가 재밌는 이유를 잘 설명해줍니다.
이 모델로 무엇을 할 수 있는지 한 가지 더 보여드리겠습니다.
한 가지 예를 더 보여드리겠습니다.
여기 농담 웹사이트의 멋진 모형을 손으로 그린 것이 있습니다.
냉장고에 붙여 놓을 만한 가치가 있습니다.
이제 휴대폰을 꺼내서
말 그대로 이 모형을 사진으로 찍어서
디스코드에 보낼 거예요.
좋아요, 디스코드에 보내볼게요. 
물론 이것이 가장 어려운 부분입니다,
우리가 실제로 올바른 채널로 보냈는지 확인하는 것입니다,
사실 잘못된 채널로 보내지 않은 것 같아요
잘못된 채널로 보낸 것 같아요.
재밌는 일이죠.
데모에서 가장 어려운 부분은 항상 AI가 아닌 부분입니다.
가장 어려운 부분이죠.
이제 시작입니다.
기술은 이제 해결되었고 이제 기다리기만 하면 됩니다.
그래서 제 마음속에서 놀라운 점은
우리가 신경망과 대화하고 있다는 점입니다.
신경망과 대화하고 있다는 것입니다.
그리고 이 신경망은 다음에 일어날 일을
다음에 일어날 일을 예측하도록 훈련되었습니다.
이 신경망은 문서의 일부를 보여주는 게임을 한 다음
다음에 무엇이 나올지 예측하는
다음 내용을 예측하는 게임을 수행했습니다.
그리고 거기서부터 이 모든 기술을 학습합니다.
매우 유연한 방식으로 적용할 수 있는 모든 기술을 학습합니다.
그래서 실제로 이 결과물을 가져올 수 있습니다.
말 그대로 저 그림에서 HTML을
HTML을 출력하라고 했습니다.
그리고 여기 있습니다.
실제 작동하는 자바스크립트가 농담을 채워 넣었습니다.
비교를 위해, 이것은 우리가 만든 목업의 원본입니다.
이렇게 손으로 그린 아름다운 예술 작품에서
제가 직접 말하자면 작동하는 웹사이트로 발전한 것입니다.
그리고 이것은 모두 잠재력일 뿐입니다.
다양한 응용 프로그램을 볼 수 있습니다.
저희도 여전히 새로운 활용 방법을 찾고 있습니다.
그래서 우리는 파트너와 협력할 것입니다.
거기서부터 확장해 나갈 것입니다.
하지만 이 기능이 제대로 작동하려면
모든 사람이 이 기능을 사용할 수 있도록 하려면
시간이 걸릴 테니까요.
마지막으로 보여드릴 것이 하나 더 있습니다.
기존 콘텐츠를 읽는 방법을 보여드렸습니다.
파트너로서 시스템과 함께 구축하는 방법을 보여드렸습니다.
마지막으로 보여드릴 것은
시스템과 협력하여 우리 중 누구도 좋아하지 않는 작업을 수행하는 방법입니다.
하지만 우리 모두는 해야만 합니다.
짐작하셨을 수도 있습니다.
우리가 할 일은 세금입니다.
GPT는 공인된 세무 전문가가 아닙니다.
따라서 항상 세무사에게 문의해야 합니다.
하지만 몇 가지 밀도 높은 내용을 이해하는 것은 도움이 될 수 있습니다.
문제를 스스로 해결할 수 있는 능력을 키우고
문제를 해결하고 무슨 일이 일어나고 있는지 파악하는 데 도움이 될 수 있습니다.
도움이 될 수 있습니다.
그래서 다시 한 번 시스템 메시지를 해보겠습니다.
이 경우에는 세금 GPT라고 말씀드리겠습니다.
우리가 이 모델에 훈련시킨 특정한 것이 아닙니다.
원한다면 시스템 메시지를 매우 창의적으로 만들 수 있습니다.
모델에게 당신의 직업이 무엇인지에 대한 분위기를 조성하고 싶으신가요?
무엇을 해야 하나요?
그래서 세금 코드를 붙여 넣었습니다.
약 16페이지 분량의 세금 코드입니다.
그리고 앨리스와 밥에 관한 질문이 있어요.
그들은 한때 결혼했습니다.
여기 그들의 소득이 있고 표준 공제를 받습니다.
그들은 공동으로 세금 신고를 하고 있습니다.
첫 번째 질문은 2018년 표준 공제액이 얼마인가요?
모델이 작동하는 동안 이 문제를 해결해 보겠습니다.
문제를 직접 풀어보겠습니다.
표준 공제는 기본 표준 공제입니다,
추가 공제입니다.
기본 공제는 하위 단락 C의 공동 환급에 대해 200%입니다,
여기에 있습니다.
좋아, 그래서 추가는 적용되지 않습니다.
한도가 적용되지 않습니다.
아니요, 적용됩니다.
오, 잠깐만요.
2018년 과세 연도에 대한 특별 규칙이 적용됩니다.
12,000달러를 3,000달러로 대체해야 합니다.
따라서 12,000의 200%인 24,000이 최종 정답입니다.
보시다시피 모델도 같은 결론을 내렸습니다.
그리고 실제로 그 설명을 읽을 수 있습니다.
그리고 진실을 말하자면, 처음으로
저는이 문제에 직접 접근하려고했습니다,
알아낼 수 없었습니다.
세금 코드를 읽는 데 30 분을 보냈습니다,
이 백 레퍼런스를 알아내려고
왜 하위 단락이 있는지 알아내려고 30분 동안 읽었습니다.
도대체 무슨 일이 벌어지고 있는 걸까요?
결국 모델에게 그 이유를 설명해 달라고 요청해야만 했습니다.
그리고 나서 저도 따라했어요.
이제야 이해가 되더군요.
어떻게 작동하는지 알겠더라고요.
이 시스템의 강점은 바로 여기에 있다고 생각합니다.
완벽하지는 않지만 여러분도 마찬가지입니다.
그리고 함께 이 증폭 도구를 사용하면
새로운 차원에 도달할 수 있습니다.
그리고 더 나아갈 수 있습니다.
이제 총 생존 가능성을 계산해 보세요.
이제 계산이 시작됩니다.
계산이 시작됩니다.
솔직히 계산을 할 때마다 놀랍습니다.
이 모델은 정신적 수학을 정말 잘합니다.
제가 하는 것보다 훨씬 더 잘해요.
계산기에 연결되어 있지 않아요.
이 시스템을 개선할 수 있는 또 다른 방법은
이러한 시스템을 개선할 수 있습니다.
하지만 이런 원시적인 기능이
매우 유연합니다.
코드든 언어든 상관없습니다.
언어도 상관없습니다.
세금도 상관없습니다.
이러한 모든 기능을 하나의 시스템에서
관심 있는 문제에 적용할 수 있습니다,
애플리케이션, 구축하는 모든 것에 적용할 수 있습니다.
마지막으로 제가 보여드릴 것은
창의력을 조금 더 발휘하는 것입니다,
이 문제를 운율 있는 시로 요약하는 것입니다.
이제 세금 납부에 관한 아름다운 시를
세금 납부에 관한 아름다운 시입니다.
시청해주신 모든 분들께 감사드립니다.
이 모델이 무엇을 할 수 있는지, 어떻게 작동하는지
어떻게 활용할 수 있는지요.
그리고 솔직히 저희는 여러분이 무엇을 만들게 될지
정말 기대됩니다. 
개안 평가에 대해 이야기했습니다.
여러분의 의견을 부탁드립니다.
저희는 이 모델을 개선하고
다음 단계로 끌어올리는 것은
모두가 기여할 수 있는 일이라고 생각합니다.
그리고 많은 사람들에게 정말 도움이 될 수 있다고 생각합니다,
여러분의 도움이 필요합니다.
정말 감사합니다.
여러분이 무엇을 만들게 될지 정말 기대됩니다.
감사합니다.

 

 

SMALL

[ English Summary ]

GPT-4 developer demo livestream. OpenAI has been working on this technology since the company was founded, but the last two years have been focused on training models, identifying risks, testing in real-world scenarios with partners, and optimizing models to bring GPT-4 to market. In this demo, we'll show you the possibilities and limitations of GPT-4, where we're still working, and how to get the most out of GPT-4 as a good tool and partner. If you'd like to join the demo stream, please join openAI's discord and give us your feedback. First, we'll showcase what GPT-4 can do that it couldn't do in 3.5, and we'll continue to do the things we do during training.

GPT-4 is an AI model under development at openAI that can do more than its predecessor. One of the tasks that GPT-4 can perform is chatbots. You can use GPT-4's API to input user messages, and the model will return appropriate responses. This makes the chatbot conversation more structured and the model more reliable. We will continue to leverage these technologies to create more advanced AI models.

GPT-4 also has the ability to create summaries that start with specific words in an article.

GPT-4 can be used to creatively utilize existing content and create AI programming assistants. OpenAI provides an open-source evaluation framework for evaluating the performance of GPT-4, which can help you identify the model's limitations and make improvements.

GPT-4 models utilize documentation to synthesize new content and handle errors that may occur in the Discord API and Jupyter Notebook. The model has the ability to utilize previously learned information in new ways to generate new content, and it also helps to handle errors that may occur in the Discord API and Jupyter Notebook. Users should always review the code generated by the model and avoid running untrusted code. Also, be aware that if the versions of the Discord API and Jupyter Notebook are different from the versions the model has been trained on, the code generated by the model may not work. To handle these errors, you can enter an error message to the model so that it can correct the error and generate the correct code. We also recommend that you install the appropriate libraries when using Jupyter Notebook to ensure that the model runs in your environment, and follow the methods suggested by the model to resolve version issues with the Discord API.

GPT-4 models can recognize images. For example, you can enter a mix of images and text through the Discord API. In order for the model to recognize the image, you must provide a detailed description of the image, and the model will analyze the image based on the information provided.
Discord requires an intent field to recognize the content of the message, which the model can use to recognize the message. However, this feature may be beyond the scope of what the model can recognize, so it needs to be optimized through testing. 

[ English Full Text ]

The GPT-4 developer demo live stream.
Honestly, it's hard for me to believe that this day is here.
OpenAI has been building this technology
really since we started the company,
but for the past two years,
we've been really focused on delivering GPT-4.
That started with rebuilding our entire training stack,
actually training the model,
and then seeing what it was capable of,
trying to figure out its capabilities,
its risks, working with partners in
order to test it in real-world scenarios,
really tuning its behavior,
optimizing the model, getting it
available so that you can use it.
Today, our goal is to show you a little bit of how to make GPT-4 shine,
how to really get the most out of it,
where its weaknesses are,
where we're still working on it,
and just how to really use it as a good tool, a good partner.
So if you're interested in participating in the stream,
that if you go to our Discord,
that's discord.gg slash openAI,
there's comments in there and we'll take
a couple of audience suggestions.
So the first thing I want to show you is the first task
that GPT-4 could do that we never really got 3.5 to do.
The way to think about this is all throughout training,
that you're constantly doing all this work.
It's 2 AM, the pager goes off,
you fix the model, and you're always wondering,
is it going to work?
Is all of this effort actually going to pan out?
So we all had a pet task that we really liked,
and that we would all individually be trying to see,
is the model capable of it now?
I'm going to show you the first one that we had a success for four,
but never really got there for 3.5.
So I'm just going to copy the top of our blog post from today,
going to paste it into our Playground.
Now this is our new chat completions Playground
that came out two weeks ago.
I'm going to show you first with GPT-3.5,
4 has the same API to it, the same Playground.
The way that it works is you have a system message
where you explain to the model what it's supposed to do,
and we've made these models very steerable,
so you can provide it with really any instruction you want,
whatever you dream up.
And the model will adhere to it pretty well,
and in the future it will get increasingly,
increasingly powerful at steering the model very reliably.
You can then paste whatever you want as a user,
the model will return messages as an assistant.
And the way to think of it is that we're moving away
from sort of just raw text in, raw text out,
where you can't tell where different parts of the conversation
come from, but towards this much more structured format
that gives the model the opportunity to know,
well, this is the user asking me to do something
that the developer didn't attend,
I should listen to the developer here.
All right, so now time to actually show you
the task that I'm referring to.
So everyone's familiar with, summarize this,
and say article into a sentence, getting a little more
specific, but where every word begins with g.
So this is 3.5, let's see what it does.
Yeah, it kind of didn't even try, just gave up on the task.
This is pretty typical for 3.5 trying
to do this particular kind of task.
If it's sort of a very kind of stilted article or something
like that, maybe it can succeed, but for the most part,
3.5 just gives up.
But let's try the exact same prompt, the exact same system
message in GPT-4.
So kind of borderline, whether you want to count AI or not.
But so let's say AI doesn't count, that's cheating.
So fair enough, the model happily accepts my feedback.
So now to make sure it's not just good for Gs,
I'd like to turn this over to the audience.
I'll take a suggestion on what letter to try next.
In the meanwhile, while I'm waiting for our moderators
to pick the lucky, lucky letter, I will give a try with A.
But in this case, I'll say GPT-4 is fine.
Why not?
Also, pretty good summary.
So I'll hop over to our Discord.
All right, wow.
If people are being a little ambitious here,
I'm really trying to put the model through the paces.
We're going to try Q, which, if you think about this
for a moment, I want the audience to really think about,
how would you do a summary of this article that
all starts with Q?
It's not easy.
It's pretty good.
That's pretty good.
All right, so I've shown you summarizing an existing article.
I want to show you how you can flexibly combine ideas
between different articles.
So I'm going to take this article that
was on Hacker News yesterday, copy-paste it,
and do the same conversation so it has all the context of what
we were just doing.
I'm going to say, find one common theme
between this article and the GPT-4 blog.
So this is an article about Pinecone, which is a Python web
app development framework, and it's
making the technology more accessible, user-friendly.
If you don't think that was insightful enough,
you can always give some feedback and say,
that was not insightful enough.
Please.
No, I'll just even just leave it there.
Leave it up to the model to decide.
So bridging the gap between powerful technology
and practical applications seems not bad.
And of course, you can ask for any other kind of task
you want using its flexible language understanding
and synthesis.
You can ask for something like, now turn the GPT-4 blog post
into a rhyming poem.
Picked up on opening IE valves, open source for all,
helping to guide answering the call, which by the way,
if you'd like to contribute to this model,
please give us E-Vals.
We have an open source evaluation framework
that will help us guide and all of our users understand
what the model is capable of and to take it to the next level.
So there we go.
This is consuming existing content using GPT-4
with a little bit of creativity on top.
But next, I want to show you how to build with GPT-4, what
it's like to create with it as a partner.
And so the thing we're going to do
is we're going to actually build a Discord bot.
I'll build it live and show you the process, show you debugging,
show you what the model can do, where its limitations are,
and how to work with them in order to achieve new heights.
So the first thing I'll do is tell the model
that this time it's supposed to be an AI programming assistant.
Its job is to write things out in pseudocode first
and then actually write the code.
And this approach is very helpful
to let the model break down the problem into smaller pieces.
And then that way, you're not asking
it to just come up with a super hard solution to a problem
all in one go.
It also makes it very interpretable,
because you can see exactly what the model was thinking,
and you can even provide corrections if you'd like.
So here is the prompt that we're going to ask it.
This is the kind of thing that 3.5 would totally
choke on if you tried anything like it.
But so we're going to ask for a Discord bot that
uses the GPT-4 API to read images and text.
Now, there's one problem here, which is this model's training
cutoff is in 2021, which means it has not seen our new chat
completion format.
So I literally just went to the blog post from two weeks ago,
copy pasted from the blog post, including the response format.
It has not seen the new image extension to that.
And so I just kind of wrote that up in just very minimal detail
about how to include images.
And now the model can actually leverage that documentation
so that it did not have memorized, that it does not know.
And in general, these models are very good at using information
that it's been trained on in new ways
and synthesizing new content.
And you can see that right here, that it actually wrote
an entirely new bot.
Now, let's actually see if this bot is
going to work in practice.
So you should always look through the code
to get a sense of what it does.
Don't run untrusted code from humans or from AIs.
And one thing to note is that the Discord API
has changed a lot over time.
And particularly, that there's one feature that
has changed a lot since this model was trained.
Give it a try.
In fact, yes, we are missing the intense keyword.
This is something that came out in 2020.
So the model does know it exists,
but it doesn't know which version of the Discord API
we're using.
So are we out of luck?
Well, not quite.
We can just simply paste to the model exactly the error
message, not even going to say, hey,
this is from running your code.
Could you please fix it?
We'll just let it run.
And the model says, oh, yeah.
Whoops, the intense argument.
Here's the correct code.
Now, let's give this a try, once again kind of making sure
that we understand what the code is doing.
Now, a second issue that can come up
is it doesn't know what environment I'm running in.
And if you notice, it says, hey, here's
this inscrutable error message, which, if you've not used
Jupyter Notebook a lot with async.io before, you probably
have no idea what this means.
But fortunately, once again, you can just sort of say to the
model, hey, I'm using Jupyter.
I'm using Jupyter, and would like to make this work, and
you fix it.
And the specific problem is that there's already an event
loop running, so you need to use this nest async.io
library.
You need to call nest async.io.apply.
The model knows all of this.
Correctly instantiates all of these pieces into the bot.
It even helpfully tells you, oh, you're running in Jupyter.
Well, you can do this bang, pip install in order to install
the package if you don't already have it.
That was very helpful.
Now we'll run, and it looks like something happened.
So the first thing I'll do is go over to our Discord, and I
will paste in a screenshot of our Discord itself.
So remember, GPT-4 is not just a language model.
It's also a vision model.
In fact, it can flexibly accept inputs that
intersperse images and text arbitrarily, kind of like a
document.
Now, the image feature is in preview.
So this is going to be a little sneak peek.
It's not yet publicly available.
It's something we're working with one partner called
Be My Eyes in order to really start to develop it and get
it ready for prime time.
But you can ask anything you like.
For example, I can't, I'll say, GPT-4, hello world, can
you describe this image in painstaking detail?
All right, which, first of all, think of how you would do
this yourself.
There's a lot of different things you could latch onto, a
lot of different pieces of the system you could describe.
And we can go over to the actual code, and we can see
that, yep, we, in fact, received the message, have
formatted an appropriate request for our API.
And now we wait, because one of the things we have to do is
we have to make the system faster.
That's one of the things that we're working on optimizing.
In the meanwhile, I just want to say to the audience that's
watching, we'll take an audience request next.
So if you have an image and a task you'd like to
accomplish, please submit that to the Discord.
Our moderators will pick one that we'll run.
So we can see that the Discord, oh, it looks like we have a
response.
Perfect.
So it's a screenshot of a Discord application interface.
Pretty good.
Did not even describe it.
It knows that it's Discord.
It's probably Discord written there somewhere where it just
kind of knows this from prior experience.
Server icon labeled GPT-4 describes the interface in
great detail.
Talks about all the people telling me that I'm supposed to
do queue.
Very kind audience.
And describes a bunch of the notification messages and the
users that are in the channel.
And so there you go.
That's some pretty good understanding.
Now this next one, if you notice, first of all, we got
a post, but the model did not actually see the message.
So is this a failure of the model or of the system around
the model?
Well, we can take a look.
And if you notice, here, content is an empty string.
We received a blank message contents.
The reason for this is a dirty trick that we
played on the AI.
So if you go to the Discord documentation and you scroll
through it, all the way down to the message content intent.
You'll see this was added as of September 2022 as a
required field.
So in order to receive a message that does not explicitly
tag you, you now have to include this new intent in
your code.
Remember I said, intents have changed a lot over time.
This is much newer than the model is
possibly able to know.
So maybe we're out of luck.
We have to debug this by hand.
But once again, we can try to use GPD4's language
understanding capabilities to solve this.
Now, keep in mind, this is a document of, I think this is
like 10,000, 15,000 words, something like that.
It's not formatted very well.
This is literally a command A copy paste.
This is what it's supposed to parse through defined in the
middle of that document that, oh yeah, message contents.
That's required now.
But let's see if it can do it.
So we will ask for, I am receiving blank message
contents.
Can you, why could this be happening?
How do I fix it?
So one thing that's new about GPD4 is context length.
32,000 tokens is kind of the upper limit that we
support right now.
And the model is able to flexibly use long documents.
It's something we're still optimizing.
So we recommend trying it out, but not necessarily really
scaling it up just yet, unless you have an application
that really benefits from it.
So if you're really interested in long context, please let
us know.
We want to see what kinds of applications it unlocks.
But if you see, it says, oh yeah, message content intent
wasn't unenabled.
And so you can either ask the model to write some code for
you, or you could actually just do it the old fashioned
way.
Either way is fine.
I think that this is an augmenting tool, makes you much
more productive.
But it's still important that you are in the driver's seat,
and are the manager, and knows what's going on.
So now we're connected once again.
And Boris, would you like to rerun the message?
Once again, we can see that we have received it, even though
the bot was not explicitly tagged.
Seems like a pretty good description.
Interesting.
This is an interesting image, actually.
It looks like it's a dolly generated one.
And let's actually try this one as well.
What's funny about this image?
Oh, it's already been submitted.
Once again, we can verify that it's making the right API
calls.
Squirrels do typically eat nuts.
We don't expect them to use a camera or act like a human.
So I think that's a pretty good explanation of why that
image is funny.
So I'm going to show you one more example of what you can do
with this model.
So I have here a nice hand-drawn mockup of a joke website.
Definitely worthy of being put up on my refrigerator.
So I'm just going to take out my phone,
literally take a photo of this mockup,
and then I'm going to send it to our Discord.
All right, going to send it to our Discord.
This is, of course, the rockiest part,
making sure that we actually send it to the right channel,
which, in fact, I think maybe I did not sent it
to the wrong channel.
It's funny.
It's always the sort of non-AI parts of these demos
that are the hardest part to do.
And here we go.
Technology is now solved, and now we wait.
So the thing that's amazing in my mind
is that what's going on here is we're
talking to a neural network.
And this neural network was trained
to predict what comes next.
It played this game of being shown a partial document
and then predicted what comes next
across an unimaginably large amount of content.
And from there, it learns all of these skills
that you can apply in all of these very flexible ways.
And so we can actually take now this output.
So literally, we just said to output the HTML
from that picture.
And here we go.
Actual working JavaScript filled in the jokes.
For comparison, this was the original of our mockup.
And so there you go, going from hand-drawn, beautiful art,
if I do say so myself, to working website.
And this is all just potential.
You can see lots of different applications.
We ourselves are still figuring out new ways to use this.
So we're going to work with our partner.
We're going to scale it from there.
But please be patient, because it's
going to take us some time to really make this
available for everyone.
So I have one last thing to show you.
I've shown you reading existing content.
I've shown you how to build with the system as a partner.
The last thing I'm going to show is
how to work with the system to accomplish a task that none of
us like to do, but we all have to.
So you may have guessed.
The thing we're going to do is taxes.
Now note that GPT is not a certified tax professional
nor am I, so you should always check with your tax advisor.
But it can be helpful to understand some dense content
to just be able to empower yourself to be able to solve
problems and get a handle on what's happening when you
could not otherwise.
So once again, I'll do a system message.
In this case, I'm going to tell it that it's tax GPT, which
is not a specific thing that we've trained into this model.
You can be very creative if you want with the system message
to really get the model in the mood of what is your job?
What are you supposed to do?
So I've pasted in the tax code.
This is about 16 pages worth of tax code.
And there's this question about Alice and Bob.
They got married at one point.
And here are their incomes and they take a standard deduction.
They're filing jointly.
So first question, what is their standard deduction for 2018?
So while the model is chugging, I'm going to solve this
problem by hand to show you what's involved.
So the standard deduction is the basic standard deduction,
plus the additional.
The basic one is 200% for joint return of sub-paragraph C,
which is here.
OK, so additional doesn't apply.
The limitation doesn't apply.
OK, no, these apply.
Oh, wait.
Special rules for taxable year 2018, which is the one we care
about through 2025, you have to substitute 12,000 for 3,000.
So 200% of 12,000, 24,000 is the final answer.
If you notice, the model got the same conclusion.
And you can actually read through its explanation.
And to tell you the truth, the first time
I tried to approach this problem myself,
I could not figure it out.
I spent half an hour reading through the tax code,
trying to figure out this back reference
and why there's sub-paragraph.
Just what's even going on?
It was only by asking the model to spell out its reasoning.
And then I followed along.
And I was like, oh, I get it now.
I understand how this works.
And so that, I think, is where the power of this system lies.
It's not perfect, but neither are you.
And together, it's this amplifying tool
that lets you just reach new heights.
And you can go further.
You can say, OK, now calculate their total viability.
And here we go.
It's doing the calculation.
Honestly, every time it does it, it's amazing.
This model is so good at mental math.
It's way, way better than I am at mental math.
It's not hooked up to a calculator.
That's another way that you could really
try to enhance these systems.
But it has these raw capabilities
that are so flexible.
It doesn't care if it's code.
It doesn't care if it's language.
It doesn't care if it's tax.
All of these capabilities in one system
that can be applied towards the problem that you care about,
towards your application, towards whatever you build.
And so to end it, the final thing that I will show
is a little other dose of creativity,
which is now summarize this problem into a rhyming poem.
And there we go, a beautiful, beautiful poem
about doing your taxes.
So thank you, everyone, for tuning in.
I hope you learned something about what the model can do,
how to work with it.
And honestly, we're just really excited to see
what you're going to build.
I've talked about open eye evals.
Please contribute.
We think that this model, improving it,
bringing it to the next level, is something
that everyone can contribute to.
And that we think it can really benefit a lot of people,
and we want your help to do that.
So thank you very much.
We're so excited to see what you're going to build.
Thank you.